# # robots.txt # # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # # This file will be ignored unless it is at the root of your host: # Used: http://example.com/robots.txt # Ignored: http://example.com/site/robots.txt # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/robotstxt.html User-agent: * Crawl-delay: 5 # Directories Disallow: */includes/ Disallow: */misc/ Allow: */misc/*.gif Allow: */misc/*.jpg Allow: */misc/*.jpeg Allow: */misc/*.png Disallow: */modules/ Disallow: */scripts/ Disallow: */themes/ Allow: */sites/all Allow: */sites/default Disallow: */profiles/commerce_kickstart/themes/* # Files Disallow: */CHANGELOG.txt Disallow: */cron.php Disallow: */*.txt Disallow: */install.php Disallow: */update.php Disallow: */xmlrpc.php # Paths (clean URLs) Disallow: */admin/ Disallow: */comment/reply/ Disallow: */filter/tips/ Disallow: */node/add/ Disallow: */search/ Disallow: */user/password/ Disallow: */user/logout/ Disallow: */node/* Disallow: */checkout* Disallow: */cart$ # Paths (no clean URLs) Disallow: /*?q=admin Disallow: /*?q=comment/reply Disallow: /*?q=filter/tips Disallow: /*?q=node/add Disallow: /*?q=search Disallow: /*?q=user # Disallow: /*?q=user/password # Disallow: /*?q=user/register/ # Disallow: /*?q=user/login/ # Disallow: /*?q=user/logout/ Disallow: */register?* # Desactivar rastreo de resultados de búsqueda Disallow: /*sort= Disallow: /*order= Disallow: /*search_api_views_fulltext=* Disallow: */search-results Disallow: */productos Allow: */productos-* Allow: */productos-alimentacion Allow: */productos-limpieza Allow: */productos-para-prp # Una vez optimizadas las marcas, cambiar por: Disallow: /*field_product_category* Disallow: /*field_subcategor_a* Disallow: /*field_marca* Disallow: */tipo/ Disallow: */type/ Disallow: */tags Disallow: */proveedor Disallow: */provider Disallow: */supplier Disallow: */newsletter/ Disallow: */return-reason Disallow: */return-reason-method Disallow: */comment Allow: */comment-* Allow: */blog/comment Disallow: */slideshow Disallow: */pregunta Allow: */preguntas Disallow: */question Allow: */questions Disallow: */respuesta Disallow: */answer Disallow: */user* # Commercial bot (GoodBot) User-agent: SemrushBot User-agent: SemrushBot-SA User-agent: dotbot User-agent: Ezooms User-agent: AhrefsBot User-agent: 1001FirmsBot User-agent: Siteliner User-agent: Cliqzbot User-agent: coccocbot-web User-agent: AhrefsBot Allow: /es/ Disallow: /es/* Allow: /en/ Disallow: /en/* Allow: /fr/ Disallow: /fr/* Allow: /it/ Disallow: /it/* Allow: /pt/ Disallow: /pt/* # Russia largest search engine spider # User-agent: Yandex # China largest search engine spider # User-agent: Baidu # Allow: / # Allow: /en/ # Allow: /fr/ # Allow: /it/ # Disallow: /* # Jobs crawling bot User-agent: Jooblebot User-Agent: MegaIndex.ru Allow: /es/ Disallow: /es/* Allow: /en/ Disallow: /en/* Allow: /fr/ Disallow: /fr/* Allow: /it/ Disallow: /it/* Allow: /pt/ Disallow: /pt/* # BAD BOT (no respeta el robots.txt) Bloqueado en .htaccess # User-agent: BLEXBot # Allow: / # Allow: /en/ # Allow: /fr/ # Allow: /it/ # Disallow: /* # DEJAR ESTAS LINEAS SIEMPRE AL FINAL Sitemap: https://www.dentaltix.com/es/sitemap.xml Sitemap: https://www.dentaltix.com/fr/sitemap.xml Sitemap: https://www.dentaltix.com/en/sitemap.xml Sitemap: https://www.dentaltix.com/it/sitemap.xml Sitemap: https://www.dentaltix.com/pt/sitemap.xml
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)