Don’t block all of your RSS feeds from search engines with robots.txt because some blog search engines won’t include your site.
I usually block all RSS feeds except for the main feed(s) on a site. I don’t know what the effect on blog search engines of blocking your Feedburner feed is (setting “noindex” through the Feedburner control panel), but recently I’ve been unblocking it.
Google recently said that they are going to stop indexing RSS feeds (first seen here), but I’ve still seen RSS feeds in the search results as recently as a few days ago.
When Google showed Supplemental Results, XML feeds would often be indexed in the Supplemental Results which made me think that Google considers them to be low quality, duplicate pages.
How do you handle indexing of your RSS/ATOM feeds? Do you think that search engines consider XML pages (RSS/ATOM feeds) to be duplicate content of the HTML pages?
One Trackback
[…] sure that your RSS/ATOM feed is not blocked to search engines, specifically Google. Don’t block your main feed with robots.txt or with Feedburner’s […]