User-agent: * Disallow: */searchterm* Disallow: */sorting* Disallow: */article/replacementinfo* Disallow: */nofollowlink Disallow: */getarticletexts* Disallow: */downloadfile* Crawl-delay: 10 Request-rate: 1/10 Visit-time: 2200-0700 User-Agent: panscient.com Disallow: / User-agent: MJ12bot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: dotbot Disallow: / User-agent: PetalBot Disallow: / User-agent: mojeekbot Disallow: / User-agent: Barkrowler Disallow:/ User-agent: Sogou web spider Disallow: / User-agent: Yandex Disallow: / User-agent: AhrefsBot Disallow: / User-agent: Bytespider Disallow: / User-agent: SEBot-WA Disallow: / User-agent: UptimeRobot/2.0 Disallow: / User-agent: SeznamBot Disallow: / User-agent: OpenindexSpider Disallow: / User-agent: YisouSpider Disallow: / User-agent: Y!J-ASR/1.0 Disallow: / User-Agent: crawler_eb_germany_2.0 Disallow: / User-Agent: omgilibot Disallow: / User-Agent: omgili Disallow: / sitemap: https://www.fristads.com/sitemap-index.xml