Die Seite ist nicht abrufbar

Diese Seite ist für den Seobility Bot nicht erreichbar. Unser Bot wird von der Robots.txt Datei ausgeschlossen. Der Seobility Bot crawlt nichts was Seobility oder * ausschließt.

Problematische URL: http://catalogue.libraries.london.ac.uk

# This file instructs all WWW robots NOT to index pages that begin 
# with the URLS listed.
User-agent: *
Disallow: /

# For the WebBridge Google Scholar Extension. Allows googlebot_IA to crawl
# /screens
User-agent: Googlebot-IA
Disallow: /acquire
Disallow: /airpac
Disallow: /airwkst
Disallow: /articles
Disallow: /availlim
Disallow: /bookill
Disallow: /bookit
Disallow: /circhistlim
Disallow: /circpix
Disallow: /cisti_order
Disallow: /clearhist
Disallow: /documents
Disallow: /donate
Disallow: /extlang
Disallow: /feeds
Disallow: /ftlist
Disallow: /goto
Disallow: /iii
Disallow: /ill
Disallow: /illframe
Disallow: /indexsort
Disallow: /journill
Disallow: /kids
Disallow: /launch
Disallow: /logout
Disallow: /manage
Disallow: /manual
Disallow: /metafind
Disallow: /mfgo
Disallow: /netli
Disallow: /nonret
Disallow: /patroninfo
Disallow: /programs
Disallow: /record=
Disallow: /review
Disallow: /search
Disallow: /selfreg
Disallow: /setlang
Disallow: /setscope
Disallow: /suggest
Disallow: /tmp
Disallow: /validate
Disallow: /VERIFYPATRON
Disallow: /VERSION
Disallow: /weblang
Disallow: /wm
Disallow: /xrecord=
Disallow: /z39
Disallow: /z39m

User-agent: msnbot
Crawl-delay: 5

# block SemrushBo crawlers
User-agent: SemrushBot
Disallow: /
Disallow: /search

User-agent: SemrushBot/7~bl; 
Diasllow: /

User-agent: GPTBot
Disallow: /

User-agent: ByteSpider
Disallow: /

User-agent: BitSiteBot
Disallow: /

#PetalBot;+https://aspiegel.com/petalbot
User-agent: PetalBot
Disallow: /

# block Yandex crawler
User-agent: Yandex
Disallow: /

# block BlexBot crawler
User-agent: BLEXBot
Disallow: /

User-agent: ClaudeBot 
Disallow: /

User-agent: Amazonbot
Disallow: /

User-agent: facebookexternalhit/1.1 
Disallow: /

User-agent: Applebot/0.1
Disallow: /

User-agent: Googlebot/2.1
Disallow: /

User-agent: YandexBot/3.0
Disallow: /

User-agent: SemrushBot/7~bl
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: SeekportBot
Disallow: /

User-agent: Googlebot/2.1
Disallow: /

User-agent: GoogleOther
Disallow: /

User-agent: GPTBot/1.2
Disallow: /

User-agent: GPTBot/1.0
Disallow: /

User-agent: DotBot/1.2
Disallow: /

User-agent: ClaudeBot/1.0
Disawllow: /

User-agent: BitSightBot/1.0
Disawllow: /