Crawl-delay: 5 User-agent: Googlebot Allow: / Crawl-delay: 5 User-agent: bingbot Crawl-delay: 10 User-agent: msnbot Crawl-delay: 10 #Block bots User-agent: A6-Indexer Disallow: / User-agent: AhrefsBot Disallow: / User-agent: AlphaSeoBot Disallow: / User-agent: AlphaSeoBot-SA Disallow: / User-agent: Applebot Disallow: / User-agent: AspiegelBot Disallow: / User-agent: barkrowler Disallow: / User-agent: Blackboard Safeassign Disallow: / User-agent: BLEXBot Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: crawler4j Disallow: / User-agent: DotBot Disallow: / User-agent: Facebot Disallow: / User-agent: facebookexternalhit Disallow: / User-agent: Gigabot Disallow: / User-agent: grapeshot Disallow: / User-agent: GPTBot Disallow: / User-agent: GumGum-Bot Disallow: / User-agent: hyscore Disallow: / User-agent: LieBaoFast Disallow: / User-agent: MauiBot Disallow:/ User-agent: MauiBot ([email protected]) Disallow: / User-agent: MJ12bot Disallow: / # Seems to ignore rules User-agent: MegaIndex User-agent: MegaIndex.ru/2.0 User-agent: MegaIndex.ru/ User-agent: +https://www.megaindex.ru/ User-agent: Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler) Disallow: / User-agent: MqqBrowser Disallow:/ User-agent: netseer Disallow: / User-agent: Nimbostratus-Bot/v1.3.2 Disallow: / User-agent: PetalBot Disallow: / User-agent: proximic Disallow: / User-agent: Qwant-news Disallow: / User-agent: Qwantify Disallow: / User-agent: seekport User-agent: Seekport Crawler Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: Semrushbot-SI Disallow: / User-agent: SEOkicks Disallow: / User-agent: SeznamBot Disallow: / User-agent: sogou spider Disallow: / User-agent: SputnikBot/2.3 Disallow:/ User-agent: The Knowledge AI Disallow:/ User-agent: TinyTestBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: UCBrowser Disallow: / User-agent: Twitterbot Disallow: / User-agent: yacybot Disallow: / User-agent: Yandex Disallow: / User-agent: YandexBot Disallow: / User-agent: Yeti Disallow: / User-agent: YisouSpider Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: DataForSeoBot Disallow: /
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)