This site can not be crawled by the Seobility bot. The robots.txt file restricts access for our bot. No URLs can be crawled that deny access to the user agent Seobility or *.
Problem URL: http://rundfunkunion.de
## CRAWLING ALLOWED ONLY FOR ## https://www.keycdn.com/blog/web-crawlers ########################################### User-agent: AdsBot-Google User-agent: AdsBot-Google-Mobile User-agent: AdsBot-Google-Mobile-Apps User-agent: Feedfetcher-Google User-agent: Google Favicon User-agent: Googlebot User-agent: Googlebot-Image User-agent: Googlebot-Mobile User-agent: Googlebot-News User-agent: Googlebot-Video User-agent: Google-Read-Aloud User-agent: APIs-Google User-agent: Bingbot User-agent: BingPreview User-agent: MediaPartners User-agent: Mediapartners-Google User-agent: MMSNBOT_Mobile User-agent: msnbot User-agent: msnbot-media User-agent: adidxbot User-agent: baiduspider User-agent: baiduspider-image User-agent: baiduspider-mobile User-agent: baiduspider-news User-agent: baiduspider-video User-agent: DuckDuckBot User-agent: DuplexWeb-Google User-agent: Exabot User-agent: facebot User-agent: FAST-WebCrawler User-agent: ia_archiver User-agent: Scooter User-agent: Slurp User-agent: Teoma User-agent: YahooSeeker/M1A1-R2D2 User-agent: YandexBot User-agent: yandex Allow: / Disallow: /*?nid=* User-agent: * Disallow: / ######################################## ### ### ORIGINAL DRUPAL 5.23 ### ######################################## # Crawl-delay: 10 # Directories Disallow: /includes/ Disallow: /misc/ Disallow: /modules/ Disallow: /profiles/ Disallow: /scripts/ Disallow: /sites/ Disallow: /themes/ # Files Disallow: /CHANGELOG.txt Disallow: /cron.php Disallow: /INSTALL.mysql.txt Disallow: /INSTALL.pgsql.txt Disallow: /install.php Disallow: /INSTALL.txt Disallow: /LICENSE.txt Disallow: /MAINTAINERS.txt Disallow: /update.php Disallow: /UPGRADE.txt Disallow: /xmlrpc.php # Paths (clean URLs) Disallow: /admin/ Disallow: /comment/reply/ Disallow: /contact/ Disallow: /logout/ Disallow: /node/add/ Disallow: /search/ Disallow: /user/register/ Disallow: /user/password/ Disallow: /user/login/ # Paths (no clean URLs) Disallow: /?q=admin/ Disallow: /?q=comment/reply/ Disallow: /?q=contact/ Disallow: /?q=logout/ Disallow: /?q=node/add/ Disallow: /?q=search/ Disallow: /?q=user/password/ Disallow: /?q=user/register/ Disallow: /?q=user/login/