# exclude help system from robots User-agent: * Disallow: lib/doc/rechnungen/ User-agent: * Disallow: lib/ User-agent: * Disallow: admin/ # but allow htdig to index our doc-tree Disallow: # disallow stress test user-agent: stress-agent Disallow: lib/doc/rechnungen/