247cerrajero.com - SEO Checker

Meta tags

Media list

Heading structure

Found links on this page

HTTP Response Header

Links from Wikipedia

Robots.txt

#
<br># robots.txt
<br>#
<br># This file is to prevent the crawling and indexing of certain parts
<br># of your site by web crawlers and spiders run by sites like Yahoo!
<br># and Google. By telling these "robots" where not to go on your site,
<br># you save bandwidth and server resources.
<br>#
<br># This file will be ignored unless it is at the root of your host:
<br># Used:    http://example.com/robots.txt
<br># Ignored: http://example.com/site/robots.txt
<br>#
<br># For more information about the robots.txt standard, see:
<br># http://www.robotstxt.org/robotstxt.html
<br>
<br>User-agent: *
<br>Crawl-delay: 10
<br># CSS, JS, Images
<br>Allow: /misc/*.css$
<br>Allow: /misc/*.css?
<br>Allow: /misc/*.js$
<br>Allow: /misc/*.js?
<br>Allow: /misc/*.gif
<br>Allow: /misc/*.jpg
<br>Allow: /misc/*.jpeg
<br>Allow: /misc/*.png
<br>Allow: /modules/*.css$
<br>Allow: /modules/*.css?
<br>Allow: /modules/*.js$
<br>Allow: /modules/*.js?
<br>Allow: /modules/*.gif
<br>Allow: /modules/*.jpg
<br>Allow: /modules/*.jpeg
<br>Allow: /modules/*.png
<br>Allow: /profiles/*.css$
<br>Allow: /profiles/*.css?
<br>Allow: /profiles/*.js$
<br>Allow: /profiles/*.js?
<br>Allow: /profiles/*.gif
<br>Allow: /profiles/*.jpg
<br>Allow: /profiles/*.jpeg
<br>Allow: /profiles/*.png
<br>Allow: /themes/*.css$
<br>Allow: /themes/*.css?
<br>Allow: /themes/*.js$
<br>Allow: /themes/*.js?
<br>Allow: /themes/*.gif
<br>Allow: /themes/*.jpg
<br>Allow: /themes/*.jpeg
<br>Allow: /themes/*.png
<br># Directories
<br>Disallow: /includes/
<br>Disallow: /misc/
<br>Disallow: /modules/
<br>Disallow: /profiles/
<br>Disallow: /scripts/
<br>Disallow: /themes/
<br>Disallow: /cdn-cgi/
<br># Files
<br>Disallow: /CHANGELOG.txt
<br>Disallow: /cron.php
<br>Disallow: /INSTALL.mysql.txt
<br>Disallow: /INSTALL.pgsql.txt
<br>Disallow: /INSTALL.sqlite.txt
<br>Disallow: /install.php
<br>Disallow: /INSTALL.txt
<br>Disallow: /LICENSE.txt
<br>Disallow: /MAINTAINERS.txt
<br>Disallow: /update.php
<br>Disallow: /UPGRADE.txt
<br>Disallow: /xmlrpc.php
<br># Paths (clean URLs)
<br>Disallow: /admin/
<br>Disallow: /comment/reply/
<br>Disallow: /filter/tips/
<br>Disallow: /node/add/
<br>Disallow: /search/
<br>Disallow: /user/register/
<br>Disallow: /user/password/
<br>Disallow: /user/login/
<br>Disallow: /user/logout/
<br># Paths (no clean URLs)
<br>Disallow: /?q=admin/
<br>Disallow: /?q=comment/reply/
<br>Disallow: /?q=filter/tips/
<br>Disallow: /?q=node/add/
<br>Disallow: /?q=search/
<br>Disallow: /?q=user/password/
<br>Disallow: /?q=user/register/
<br>Disallow: /?q=user/login/
<br>Disallow: /?q=user/logout/
<br><br>Sitemap: https://247cerrajero.com/sitemap_index.xml

Search preview