# $Id: robots.txt,v 1.9.2.1 2008/12/10 20:12:19 goba Exp $## robots.txt## This file is to prevent the crawling and indexing of certain parts# of your site by web crawlers and spiders run by sites like Yahoo!# and Google. By telling these "robots" where not to go on your site,# you save bandwidth and server resources.## This file will be ignored unless it is at the root of your host:# Used:    http://example.com/robots.txt# Ignored: http://example.com/site/robots.txt## For more information about the robots.txt standard, see:# http://www.robotstxt.org/wc/robots.htmlUser-agent: *Crawl-delay: 10# DirectoriesDisallow: /database/Disallow: /includes/Disallow: /misc/Disallow: /modules/Disallow: /sites/Disallow: /themes/Disallow: /scripts/Disallow: /updates/Disallow: /profiles/# FilesDisallow: /xmlrpc.phpDisallow: /cron.phpDisallow: /update.phpDisallow: /install.php# Paths (clean URLs)Disallow: /admin/Disallow: /aggregatorDisallow: /comment/reply/Disallow: /contactDisallow: /logoutDisallow: /node/addDisallow: /search/Disallow: /user/registerDisallow: /user/passwordDisallow: /user/login# Paths (no clean URLs)Disallow: /?q=admin/Disallow: /?q=aggregator/Disallow: /?q=comment/reply/Disallow: /?q=contact/Disallow: /?q=logout/Disallow: /?q=node/add/Disallow: /?q=search/Disallow: /?q=user/password/Disallow: /?q=user/register/Disallow: /?q=user/login/