Sitemap: https://ryankinder.com/sitemap.xml User-agent: Twitterbot Disallow: User-agent: dataprovider Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/applebot-extended User-agent: Applebot-Extended Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/bytespider User-agent: Bytespider Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/ccbot User-agent: CCBot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/claudebot User-agent: ClaudeBot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/diffbot User-agent: Diffbot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/facebookbot User-agent: FacebookBot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/google-extended User-agent: Google-Extended Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/gptbot User-agent: GPTBot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/meta-externalagent User-agent: Meta-ExternalAgent Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/omgili User-agent: omgili Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/timpibot User-agent: Timpibot Disallow: / # Undocumented AI Agent # https://darkvisitors.com/agents/anthropic-ai User-agent: anthropic-ai Disallow: / # Undocumented AI Agent # https://darkvisitors.com/agents/claude-web User-agent: Claude-Web Disallow: / # Undocumented AI Agent # https://darkvisitors.com/agents/cohere-ai User-agent: cohere-ai Disallow: / User-Agent: * Crawl-delay: 10 Disallow: /*?auth_code=* Disallow: /api/cart/ Disallow: /purchases/ Disallow: /user_sessions/ Disallow: /go/deferred_features/ Disallow: /go/cart/ Disallow: /go/member/ Disallow: /go/members/ Disallow: /go/subscriptions/
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)