#Keep all bots out of following directories (these directories are for admin access only or #pertain to the layout of the site and do not need to be indexed. User-Agent: Googlebot-Mobile Allow: /mobile/ Allow: /iphone/ Disallow: /admin_page/ Disallow: /class/ Disallow: /images/ Disallow: /style/ Disallow: /slideshow/ User-Agent: Googlebot-Image User-Agent: msnbot-MM Allow: /images/ Disallow: /admin_page/ Disallow: /class/ Disallow: /style/ Disallow: /slideshow/ Disallow: /mobile/ User-agent: * Allow: / Allow: /mobile/mobisitemapxml.php Allow: /mobile/mobisitemap.php Allow: /iphone/iphonesitemapxml.php Allow: /iphone/iphonesitemap.php Disallow: /admin_page/ Disallow: /class/ Disallow: /images/ Disallow: /style/ Disallow: /slideshow/ Disallow: /mobile/ Disallow: /iphone/ Sitemap: http://www.coloradoconceptlighting.com/sitemap.php Sitemap: http://www.coloradoconceptlighting.com/sitemapxml.php ####################################################################################### # Comments: # Other available commands for robot.txt # Request-rate: p/s - allows maximum page rate x page(s) every s second(s) # Example: # Request-rate: 1/5 # maximum rate is one page every 5 seconds # Visit-time: xxxx-yyyy - only visit page between xxxx and yyyy # Example: # Visit-time: 0600-0845 # only visit between 06:00 and 08:45 UTC (GMT) # Allow: - some bots recognize allow, giving access to directory of file # Crawl-Delay: x - Delay crawl (time of access) by x seconds # Example: # Crawl-Delay: 10 # delay crawl to same server by 10 seconds # # # # #For more information on the robot.txt file or robot tags, visit robotstxt.org #########################################################################################