User-agent: * #Sitemapindex Sitemap: https://op.europa.eu/documents/d/global/sitemap_index-xml # Disallow search results of the portal Disallow: /*/search-centre* Disallow: /*/search-results* Disallow: /*/browse-by-subject* Disallow: /*/linked-data* Disallow: /*/advanced-sparql-query-editor* Disallow: /*/more-search-options* Disallow: /*/search-by-theme* # Disallow private pages (dashboards) Disallow: /*/dashboard* Disallow: /*/saved-annotations* Disallow: /*/user-confirmation* Disallow: /*/my-profile* Disallow: /*/my-personalisation-rules* Disallow: /*/my-portal* Disallow: /*/my-searches* Disallow: /*/my-alerts* Disallow: /*/my-rss-feeds* Disallow: /*/my-publications* Disallow: /*/my-addresses* Disallow: /*/my-lists* Disallow: /*/my-widgets* Disallow: /*/my-ratings* Disallow: /*/my-orders* # Logout pages Disallow: /c/portal/ # Do not index the shopping basket Disallow: /*/basket # Do not index the bookshop urls Disallow: /new-bookshop/ # Private content Disallow: /dempubweb # Action to return the rdf of a publication Disallow: *requestAction=rdf # Should not be indexed Disallow: /webdav/* # Site not released in production Disallow: /*/web/isg/*/ #Publication URLs that do not need indexation Disallow: /*prodSystem-*/ Disallow: /*/publication-detail/-/publication/*/format-* #Different publication manifestations to be revised Jan 2024 Disallow: /publication/doi/ Disallow: /publication/manifestation_identifier/ Disallow: /publication/catalogue_number/ Disallow: /lastedition/ #Disallow the public procurement pages before the project goes live TO BE REVISED Disallow: /web/public-procurement/ #Redirection to publication detail pages Disallow: /eu-law-and-publications/ # Duplicate URLs Google chose different canonical than user Disallow: /*/who-is-who/person/-/person/*/ # EU Vocabularies specific rules for decommissioned pages Disallow: /*/at-dataset/* Disallow: /*/at-concept/* Disallow: /*/at-concept-scheme/* Disallow: /*/th-dataset/* Disallow: /*/th-concept/* Disallow: /*/th-concept-scheme/* # Disallow the wrong dataset urls Disallow: /*/eu-vocabularies/dataset/-/resource?uri=http://publications.europa.eu/resource/dataset/*/ # Disallow the EU Vocabularies dataset versions Disallow: /*/eu-vocabularies/dataset/*version # Allow the browse by subject portlet Allow: *SubjectFilterPortlet # Short URL link. Must not be crawled. It returns a redirection Disallow: /s/* # Added to prevent the crawling of all permanent link pop-ups creating short url Disallow: /*shortUrl$ User-agent: Sogou web spider Disallow: / User-agent: PetalBot Disallow: / User-agent: LinkedInBot Allow: / User-agent: facebookexternalhit Allow: / User-agent: Twitterbot Disallow: User-agent: msnbot/bingbot Crawl-delay: 1
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)