# Ban rogue search engines User-agent: AhrefsBot Disallow: / User-agent: Baiduspider Disallow: / User-Agent: bingbot Disallow: / User-Agent: MJ12bot Disallow: / User-agent: Slurp Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: PetalBot Disallow: / User-agent: dotbot Disallow: / User-agent: Yeti Disallow: / User-agent: MojeekBot Disallow: / User-agent: serpstatbot Disallow: / # we allow the 'timeline' and 'downloads' pages itself, # but not with params. User-agent: * Disallow: /browser Disallow: /changeset Disallow: /diff Disallow: /downloads/ Disallow: /query Disallow: /log Disallow: /timeline? Crawl-delay: 60
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)