# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism User-agent: * Disallow: /apps Disallow: /dropdown Disallow: /eMembership Disallow: /events Disallow: /gsbacwo Disallow: /gsbarims Disallow: /gsbawcf Disallow: /images Disallow: /inc Disallow: /menu Disallow: /templates