# All robots will spider the domain User-agent: GoogleBot Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /junk/ User-agent: Googlebot Disallow: Crawl-delay: 60 User-agent: Adsbot-Google Disallow: Crawl-delay: 60 User-agent: Googlebot-Image Disallow: Crawl-delay: 60 User-agent: Googlebot-Mobile Disallow: Crawl-delay: 60