# See http://www.robotstxt.org/robotstxt.html for documentation User-agent: * Crawl-Delay: 5 Disallow: /account* Disallow: /addresses* Disallow: /admin* Disallow: /bisac_subjects/* Disallow: /cart* Disallow: /downloads* Disallow: /products/*/compare* Disallow: /products/advanced_search Disallow: /products/search_results* Disallow: /links* Disallow: /lists* Disallow: /marc_reports* Disallow: /quotes* Disallow: /searches* Disallow: /shop/search_results* Disallow: /specs* Disallow: /users* Disallow: /wishlists* User-agent: Adsbot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: PetalBot Disallow: / User-agent: Seekport Crawler Disallow: / User-agent: SemrushBot Disallow: / User-agent: SenutoBot Disallow: / Sitemap: https://bellwethermedia.com/sitemap.xml.gz