# Directions for web crawlers. # See http://www.robotstxt.org/wc/norobots.html. User-agent: * Disallow: /gitweb* Disallow: /git/ Disallow: /cgit/ Disallow: /changes/ Disallow: /plugins/ Disallow: /login/ User-agent: HTTrack User-agent: puf User-agent: MSIECrawler User-agent: Nutch Disallow: /