User-agent: * # directed to all robots Disallow: /cgi-bin/ Disallow: /admin/ Disallow: /_intern/ Disallow: /webEdition/ Disallow: /piwik/ Request-rate: 1/1 Crawl-delay: 1 Visit-time: 0000-0430 # Block wiederfreibot as it is just noise User-agent: wiederfreibot/1.0 Disallow: / # Block CazoodleBot as it does not present correct accept content headers User-agent: CazoodleBot Disallow: / # Block MJ12bot as it is just noise User-agent: MJ12bot Disallow: / # Block dotbot as it cannot parse base urls properly User-agent: dotbot/1.0 Disallow: / # Block Gigabot User-agent: Gigabot Disallow: / ## Allow search crawlers to discover the sitemap #Sitemap: /sitemap.xml
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)