User-agent: ia_archiver Disallow: / User-agent: * Disallow: /wp-login.php Disallow: /wp-admin/ Disallow: /wp-plugins/ Disallow: /wp-includes/ Disallow: /date Disallow: /author Disallow: /*/feed/ Disallow: /page/ Disallow: /*? Disallow: /*/trackback/ Disallow: /mi Disallow: /toolbox/ # Cleanup: # Disallow: /domains/7/feed # redirect that instead, because of penalty