User-agent: * Disallow: /rbruno Disallow: /~banon/ Disallow: /~analucia/URLib/ Disallow: /Eventos_DPI/ Disallow: /awstats/ Disallow: /consdpi/ Disallow: /tmp/ Disallow: /public/ Disallow: /quem_somos/ Disallow: /agenda/ Disallow: /horde/ Disallow: /phorum/ Disallow: /proarco/bdqueimadas/tabelatxt_uc.php/ Disallow: /OpenLayers/ Disallow: /ext-3.0.0/ Disallow: /INPE_DVD/ Disallow: /mosaico/ Disallow: /mosaico Disallow: /mosaico2/ Disallow: /collab/ Disallow: /home/mosaico/ Disallow: /home/mosaico # # $Id: robots.txt 92365 2007-09-23 14:04:27Z oden $ # $HeadURL: svn+ssh://svn.mandriva.com/svn/packages/cooker/apache-conf/current/SOURCES/robots.txt $ # exclude help system from robots User-agent: * Disallow: /manual/ Disallow: /manual-1.3/ Disallow: /manual-2.0/ Disallow: /manual-2.2/ Disallow: /addon-modules/ Disallow: /doc/ Disallow: /images/ # the next line is a spam bot trap, for grepping the logs. you should _really_ change this to something else... Disallow: /all_our_e-mail_addresses # same idea here... Disallow: /admin/ # but allow htdig to index our doc-tree #User-agent: htdig #Disallow: # disallow stress test user-agent: stress-agent Disallow: /