Techbriefs.com - SEO Checker

Metaetiquetas

Lista de medios

Estructura de los encabezados

Enlaces detectados en esta página

Cabecera HTTP

Backlinks desde Wikipedia

• Carrera espacial
• Membrana de intercambio de protones
• Susan G. Finley

Robots.txt

# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html

User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

# try to keep bots away from search
Disallow: /tb/search
Disallow: /mdb/search
Disallow: /search

User-agent: TurnitinBot
Crawl-delay: 10

User-agent: Chrome-Lighthouse
Crawl-delay: 10

#### the below needs to be added to svn
User-agent: AhrefsBot
Crawl-delay: 10

User-agent: SemrushBot-SA
Disallow: / 

User-agent: SemrushBot
Crawl-delay: 10

User-agent: bingbot
Crawl-delay: 10

# https://moz.com/help/moz-procedures/crawlers/rogerbot
User-agent: rogerbot
Crawl-delay: 10
#Disallow: /

# https://babbar.tech/crawler
User-agent: Barkrowler
Crawl-delay: 8

# https://serpstatbot.com/
User-Agent: serpstatbot
Crawl-Delay: 10

#https://megaindex.com/crawler
User-Agent: MegaIndex.ru/2.0
Crawl-Delay: 10

#https://www.brandwatch.com/legal/magpie-crawler/
User-agent: magpie-crawler
Crawl-Delay: 10

# https://www.linkdex.com/en-us/about/bots/
#User-agent: linkdexbot
#Crawl-Delay: 10

# https://dataforseo.com/dataforseo-bot
User-agent: DataForSeoBot
Crawl-delay: 10

# https://www.babbar.tech/crawler   
# see https://www.abuseipdb.com/check/154.54.249.204 
User-agent: barkrowler
Crawl-Delay: 10

# https://neeva.com/neevabot
#  100.26.127.17
User-Agent: Neevabot
Crawl-delay: 10

# https://bot.seekport.com
User-Agent: SeekportBot
Crawl-delay: 10

User-Agent: FriendlyCrawler
Crawl-delay: 10

# http://yandex.com/bots
User-Agent: YandexBot
Crawl-delay: 10

Snippet (vista previa de los resultados de búsqueda)