Claire.co.uk - SEO Check

Meta Tags

Medienliste

Überschriftenstruktur

Gefundene Links auf dieser Seite

HTTP-Header

Links von Wikipedia

Robots.txt

# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html

# Bots that use too many resources
# Huawei search bot
User-agent: PetalBot
Disallow: /

# SEMrush search bot
User-agent: SEMrushBot
Disallow: /
User-agent: SiteAuditBot
Disallow: /
User-agent: SemrushBot-BA
Disallow: /
User-agent: SemrushBot-SI
Disallow: /
User-agent: SemrushBot-SWA
Disallow: /
User-agent: SemrushBot-CT
Disallow: /
User-agent: SemrushBot-BM
Disallow: /
User-agent: SplitSignalBot
Disallow: /

#Majestic search bot
User-agent: MJ12bot
Disallow: /

# Ahref search bot
User-agent: AhrefsBot
Disallow: /

# Moz.com search bot
User-agent: DotBot
Disallow: /

# MauiBot search bot
User-agent: MauiBot
Disallow: /

User-agent: * 
Crawl-Delay: 10

User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
Disallow: /eLearning/
Disallow: /moodle/
Disallow: /phocadownload/
Disallow: /phocadownloadpap/
Disallow: /phocamapskml/
Disallow: /assessordocs/
Disallow: /certificates/
Disallow: /data/
Disallow: /media/
Disallow: /templates/
Disallow: /information-centre/
Disallow: /commerce/
Disallow: /test/
Disallow: /rom-soil-treatment-facilities/

Suchvorschau