# All robots will spider the domain User-agent: * Disallow: # Disallow following directories from being indexed User-agent: * Disallow: /cgi-local/ Disallow: /flash/ Disallow: /images/ Disallow: /novolla/ Disallow: /pdfs/ Disallow: /products/ Disallow: /scripts/ Disallow: /styles/ Disallow: /stats/