# =================================== # Website: https://essential.gg/ # =================================== Sitemap: https://essential.gg/sitemap.xml # =================================== # Allowed Crawlers # =================================== # Archive.org Configuration User-Agent: ArchiveBot Allow: * Crawl-Delay: 4 # Panascais Search Configuration User-Agent: Panascais-Search-Bot Allow: * # Google Configuration User-Agent: Googlebot User-Agent: Googlebot-Image User-Agent: Googlebot-News User-Agent: Googlebot-Video User-Agent: Googlebot-Mobile User-Agent: Google-InspectionTool User-Agent: Storebot-Google Allow: * Crawl-Delay: 1 # Bing Configuration User-Agent: Bingbot Allow: * Crawl-Delay: 1 # Yahoo! Configuration User-Agent: Slurp Allow: * Crawl-Delay: 1 # DuckDuckGo Configuration User-Agent: DuckDuckBot Allow: * Crawl-Delay: 1 # Yandex Configuration User-Agent: YandexBot Allow: * Crawl-Delay: 1 # Baidu Configuration User-Agent: Baiduspider Allow: * Crawl-Delay: 4 # Naver Configuration User-Agent: Yeti Allow: * Crawl-Delay: 4 # Qwant Configuration User-Agent: Qwantify Allow: * Crawl-Delay: 4 # Ecosia Configuration User-Agent: EcosiaBot Allow: * Crawl-Delay: 4 # Startpage Configuration User-Agent: StartpageBot Allow: * Crawl-Delay: 4 # Searx Configuration User-Agent: searx Allow: * Crawl-Delay: 4 # Kagi Configuration User-Agent: KagiBot Allow: * Crawl-Delay: 4 # =================================== # Disallowed Crawlers # =================================== # Google Extended Configuration User-Agent: Google-Extended Disallow: / # Common Crawl Configuration User-Agent: CCBot Disallow: / # OpenAI Configuration User-Agent: GPTBot User-Agent: ChatGPT-User Disallow: / # Perplexity AI Configuration User-Agent: PerplexityBot Disallow: / # Majestic 12 SEO Configuration User-Agent: MJ12bot Disallow: / # =================================== # Generic Bad Crawlers # =================================== User-Agent: WebReaper Disallow: / User-Agent: WebCopier Disallow: / User-Agent: Offline Explorer Disallow: / User-Agent: HTTrack Disallow: / User-Agent: Microsoft.URL.Control Disallow: / User-Agent: EmailCollector Disallow: / User-Agent: penthesilea Disallow: / # =================================== # Global Configuration # =================================== User-Agent: * Allow: *
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)