# Martial Arts Center München - Robots.txt # Optimierte Version basierend auf aktueller Konfiguration # Sicherheit + KI-Optimierung kombiniert # =========================================== # KI-TRAINING BOTS - GEZIELT STEUERN # =========================================== # OpenAI GPTBot (ChatGPT Training) User-agent: GPTBot Allow: /blog/ Allow: /selbstverteidigung/ Allow: /ving-tsun/ Allow: /kampfkunst/ Allow: /ueber-uns/ Allow: /programme/ Disallow: /wp-admin/ Disallow: /wp-includes/ Crawl-delay: 2 # Google Bard/Gemini Training User-agent: Google-Extended Allow: /blog/ Allow: /selbstverteidigung/ Allow: /ving-tsun/ Allow: /kampfkunst/ Allow: /ueber-uns/ Allow: /programme/ Disallow: /wp-admin/ Disallow: /wp-includes/ Crawl-delay: 2 # Common Crawl (für KI-Modelle) User-agent: CCBot Allow: /blog/ Allow: /selbstverteidigung/ Allow: /ving-tsun/ Allow: /kampfkunst/ Disallow: /wp-admin/ Crawl-delay: 3 # Anthropic Claude (falls separate Crawler) User-agent: Claude-Web Allow: / Disallow: /wp-admin/ Disallow: /wp-includes/ Crawl-delay: 2 # =========================================== # SUCHMASCHINEN - OPTIMIERT # =========================================== # Google Hauptbot User-agent: Googlebot Allow: / Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-*.php Disallow: */trackback/ Disallow: */feed/ Disallow: /search Disallow: /?. Crawl-delay: 1 # Bing User-agent: Bingbot Allow: / Disallow: /wp-admin/ Disallow: /wp-includes/ Crawl-delay: 2 # =========================================== # SOCIAL MEDIA BOTS # =========================================== User-agent: facebookexternalhit Allow: / User-agent: Twitterbot Allow: / User-agent: LinkedInBot Allow: / User-agent: WhatsApp Allow: / # =========================================== # AGGRESSIVE/UNERWÜNSCHTE BOTS # =========================================== # Deine bewährten Blockierungen User-agent: URLMetriken Disallow: / User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: PetalBot Disallow: / # Zusätzliche problematische Bots User-agent: SemrushBot-SA Disallow: / User-agent: MauiBot Disallow: / User-agent: SurdotlyBot Disallow: / User-agent: ZoomBot Disallow: / User-agent: DotBot Crawl-delay: 30 # =========================================== # ALLE ANDEREN BOTS (FALLBACK) # =========================================== User-agent: * Allow: / Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-*.php Disallow: */trackback/ Disallow: */feed/ Disallow: /search Disallow: /?. Crawl-delay: 5 # =========================================== # SITEMAP # =========================================== Sitemap: https://martial-arts-center.de/sitemap_index.xml
Se han encontrado las siguientes palabras clave. Comprueba si esta página está bien optimizada para cada palabra clave en concreto.
(Deseable)