################################# # YouPosted.com Smart Robots v2 ################################# # I have implemented a smart Robots.txt due to the incompatibility issues between different # robots and whether they support wildcards (*), multiple user-agents and end-anchors ($) # This means that I am now serving a spider-specific robots.txt to Google/Yahoo/MSNbot # and a generic one to every other bot/person. # # Detected Spider/Bot: None # Serving Robots.txt: Generic # # Headers Sent: # Content-Type: text/plain # Expires: Tue, 15 Apr 2008 06:16:21 GMT (2 day validity) # # My Sitemap - I don't provide it just for the fun of it User-agent: * # All Spiders should understand these # Default SMF Folders Disallow: /cgi-bin/ Disallow: /images/ Disallow: /attachments/ Disallow: /Packages/ Disallow: /Smileys/ Disallow: /Sources/ Disallow: /Themes/ # Default SMF Actions Disallow: /index.php?action=activate Disallow: /index.php?action=admin Disallow: /index.php?action=calendar Disallow: /index.php?action=emailuser Disallow: /index.php?action=findmember Disallow: /index.php?action=help Disallow: /index.php?action=helpadmin Disallow: /index.php?action=login Disallow: /index.php?action=logout Disallow: /index.php?action=mlist Disallow: /index.php?action=modifykarma Disallow: /index.php?action=pm Disallow: /index.php?action=post Disallow: /index.php?action=printpage Disallow: /index.php?action=profile Disallow: /index.php?action=recent Disallow: /index.php?action=register Disallow: /index.php?action=reminder Disallow: /index.php?action=search Disallow: /index.php?action=unread Disallow: /index.php?action=unreadreplies Disallow: /index.php?action=verificationcode Disallow: /index.php?action=who Disallow: /index.php?theme # Spammers - Passing links doesn't work, but stop the requests as well Disallow: /index.php?action=http Disallow: /index.php?board=http Disallow: /index.php?topic=http # Non-SMF related/Junk Disallow: /runsearch/ Disallow: /runsearch Disallow: /freelinks.php Disallow: /No%20advertising Disallow: /rc3/ Disallow: /rc3/index.php?action=links