User-agent: * Disallow: /cart/ Disallow: /login/ Disallow: /my-account/ Disallow: /orderTrack/ Disallow: /store/ Allow: /shop-custom-logo-uniforms # Allow search crawlers to discover the sitemap Sitemap: http://business.landsend.com/sitemap.xml Sitemap: https://business.landsend.com/shop-custom-logo-uniforms/sitemap.xml # Block CazoodleBot as it does not present correct accept content headers User-agent: CazoodleBot Disallow: / # Block MJ12bot as it is just noise User-agent: MJ12bot Disallow: / # Block dotbot as it cannot parse base urls properly User-agent: dotbot/1.0 Disallow: / # Block Gigabot User-agent: Gigabot Disallow: / # throttle MS bots User-agent: msnbot Disallow: /cart/ Disallow: /login/ Disallow: /my-account/ Disallow: /orderTrack/ Disallow: /store/ Crawl-delay: 1 User-agent: bingbot Allow: /shop-custom-logo-uniforms Disallow: /cart/ Disallow: /login/ Disallow: /my-account/ Disallow: /orderTrack/ Disallow: /store/ Crawl-delay: 1 User-agent: bingbot/2.0 Disallow: /cart/ Disallow: /login/ Disallow: /my-account/ Disallow: /orderTrack/ Disallow: /store/ Crawl-delay: 1 # blocking Yandex as they repeatedly hit dead URLs User-agent: YandexBot/3.0 Disallow: / User-agent: yandex Disallow: / # blocking Rogerbot User-agent: rogerbot Disallow: / # blocking SemrushBot User-agent: SemrushBot Disallow: / # blocking Commoncrawl User-agent: CCBot Disallow: /
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)