# UNRIC Robots.txt - SEO & Performance Optimized - IA Block Agent (PC release - 17 April 2025) User-agent: * # Block backend and system directories Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /cgi-bin/ Disallow: /wp-login.php Disallow: /xmlrpc.php # Block sensitive plugin directory # It's crucial to keep wp-content/plugins/ blocked to ensure robots # don't access any sensitive files that may be present, but also to reduce the load on the site. # If you are using plugins to generate content that search engines need to access, # then you need to find a different configuration for your site. Disallow: /wp-content/plugins/ # Block cache directory Disallow: /wp-content/cache/ # Block theme files (review carefully) #Disallow: /wp-content/themes/ # Only uncomment if you have very specific reasons # Block search pages and duplicate content Disallow: /?s= Disallow: /search/ Disallow: /page/*/?s= # Block WordPress duplicates and unused endpoints Disallow: */trackback/ Disallow: */feed/ Disallow: */comments/ Disallow: /*?replytocom # Block file types not meant for indexing Disallow: /*.php$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Disallow: /*.swf$ Disallow: /*.inc$ Disallow: /*.wmv$ # Allow AJAX Allow: /wp-admin/admin-ajax.php # Let Cloudflare, RocketCDN & Fonts load freely Allow: /*.css$ Allow: /*.js$ Allow: /*.woff$ Allow: /*.woff2$ Allow: /*.ttf$ Allow: /*.eot$ Allow: /*.svg$ # BLOCK AI CRAWLERS (April 2025) User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: Google-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: AnthropicAI Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Amazonbot Disallow: / # Sitemaps for Yoast SEO Multisite Sitemap: https://unric.org/sitemap_index.xml Sitemap: https://unric.org/en/sitemap_index.xml Sitemap: https://unric.org/fr/sitemap_index.xml Sitemap: https://unric.org/it/sitemap_index.xml Sitemap: https://unric.org/pt/sitemap_index.xml Sitemap: https://unric.org/nl/sitemap_index.xml Sitemap: https://unric.org/es/sitemap_index.xml Sitemap: https://unric.org/de/sitemap_index.xml Sitemap: https://unric.org/sv/sitemap_index.xml Sitemap: https://unric.org/da/sitemap_index.xml Sitemap: https://unric.org/no/sitemap_index.xml Sitemap: https://unric.org/fi/sitemap_index.xml Sitemap: https://unric.org/is/sitemap_index.xml Sitemap: https://unric.org/el/sitemap_index.xml
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)