User-Agent: YoudaoBot Disallow: / User-Agent: Sogou Disallow: / User-agent: Ocelli Disallow: /*?* Crawl-delay: 120 User-agent: * Sitemap: http://www.phys.uconn.edu/sitemap.xml.gz Disallow: /cgi-bin/ Disallow: /textonly/ Disallow: *calendarx* Disallow: /search Disallow: /intranet Disallow: /~senderovich/GlueX/ Disallow: /~he/print/ Disallow: /olympiad/Olympiad07/OlympiadO6 Disallow: /olympics/Olympiad08/OlympiadO6 Disallow: /olympiad/Olympiad08/Olympiad07 Disallow: /olympics/Olympiad08/Olympiad07 Disallow: /~omakarov Disallow: /~qpeng Disallow: /~syelin/152_f200 Disallow: /~bruce Disallow: /~chen/Paper Disallow: /~chen/paper Disallow: /~eyler/phys256 Disallow: /~rcote/cgi-bin/ Disallow: /~rcote/Courses/ Disallow: /~Rcote/Courses/ Disallow: /~RCOTE/Courses/ Disallow: /Members/crudo Disallow: /zhouran-data Disallow: /phys1501 Disallow: /phys3202 Disallow: /data/ # Googlebot allows regex in its syntax # Block all URLs including query strings (? pattern) - contentish objects # expose query string only for actions or status reports which # might confuse search results. # This will also block ?set_language User-Agent: Googlebot Disallow: /*?* Disallow: /*folder_factories$