I have noticed quite a bit of duplicated data being indexed on my Wordpress website which is going to dilute any SEO work done. I have also noticed a few directorys being indexed which is bad practice from a security point of view.
Here is what I have so far... have I missed anything or blocked somthing I shouldn't have. I really only want my main pages indexed.
Code:
User-agent: SiteSnagger
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /wp-content/uploads/lg-gallery/
Disallow: /wp-content/uploads/
Disallow: /wp-content/themes/zen/audio_player/
Disallow: /wp-login.php
Disallow: /*wp-login.php*
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /author
Disallow: /contact/
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /category/*
Disallow: /category/
Disallow: /login/
Sitemap: http://www.thirsk-homeitsolutions.co.uk/sitemap.xml.gz