############################## # robots.txt file # based on webmasterworld.com # and searchengineworld.com # Please, we do NOT allow nonauthorized robots any longer. # Yes, feel free to copy and use the following. User-agent: * Disallow: /cgi-bin/ Disallow: /asclick/ Disallow: /post_G1/ Disallow: /register_G2/ Disallow: /afs_click/ Disallow: /run/faq/ Disallow: /m/run/faq/ Disallow: /my/ Disallow: /m/my/ Disallow: /*index.rss$ Disallow: /images/sprites/ Disallow: /images/bg/ Disallow: /images/16/ Disallow: /images/32/ Disallow: /images/48/ Disallow: /images/64/ Disallow: /images/100/ Disallow: /images/128/ Disallow: /images/256/ Disallow: /images/share/ # desktop Disallow: /manage/ # mobile Disallow: /*/manage/ User-agent: msnbot Crawl-delay: 60 Disallow: /cgi-bin/ Disallow: /asclick/ Disallow: /post_G1/ Disallow: /register_G2/ Disallow: /afs_click/ Disallow: /api/ Disallow: /run/faq/ Disallow: /m/run/faq/ Disallow: /my/ Disallow: /m/my/ Disallow: /images/sprites/ Disallow: /images/bg/ Disallow: /images/16/ Disallow: /images/32/ Disallow: /images/48/ Disallow: /images/64/ Disallow: /images/100/ Disallow: /images/128/ Disallow: /images/256/ Disallow: /images/share/ # desktop Disallow: /manage/ # mobile Disallow: /*/manage/ User-agent: bingbot Crawl-delay: 60 Disallow: /cgi-bin/ Disallow: /asclick/ Disallow: /post_G1/ Disallow: /register_G2/ Disallow: /afs_click/ Disallow: /api/ Disallow: /run/faq/ Disallow: /m/run/faq/ Disallow: /my/ Disallow: /m/my/ Disallow: /*index.rss$ Disallow: /images/sprites/ Disallow: /images/bg/ Disallow: /images/16/ Disallow: /images/32/ Disallow: /images/48/ Disallow: /images/64/ Disallow: /images/100/ Disallow: /images/128/ Disallow: /images/256/ Disallow: /images/share/ # desktop Disallow: /manage/ # mobile Disallow: /*/manage/ User-agent: Slurp Crawl-delay: 60 Disallow: /cgi-bin/ Disallow: /asclick/ Disallow: /post_G1/ Disallow: /register_G2/ Disallow: /afs_click/ Disallow: /api/ Disallow: /run/faq/ Disallow: /m/run/faq/ Disallow: /my/ Disallow: /m/my/ Disallow: /*index.rss$ Disallow: /images/sprites/ Disallow: /images/bg/ Disallow: /images/16/ Disallow: /images/32/ Disallow: /images/48/ Disallow: /images/64/ Disallow: /images/100/ Disallow: /images/128/ Disallow: /images/256/ Disallow: /images/share/ # desktop Disallow: /manage/ # mobile Disallow: /*/manage/ #################################### User-agent: Mediapartners-Google Disallow: User-agent: RankSonicBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: spbot Disallow: / User-agent: A6-Indexer/1.0 Disallow: / User-agent: sistrix Disallow: / User-agent: trovitBot Disallow: / User-agent: Lipperhey Disallow: / User-agent: Sosospider Disallow: / User-agent: ia_archiver Disallow: / User-agent: ia_archiver/1.6 Disallow: / User-agent: appi Disallow: / User-agent: BoeserSeoSpider Disallow: / User-agent: StackRambler Disallow: / User-agent: Sphider Disallow: / User-agent: Jyxobot/1 Disallow: / User-agent: Jyxobot Disallow: / User-agent: gigabot Disallow: / User-agent: ccbot Disallow: / User-agent: dotbot Disallow: / User-agent: dotbot/1.0 Disallow: / User-agent: MJ12bot Disallow: / User-agent: shopwiki Disallow: / User-agent: NaverBot-1.0 Disallow: / User-agent: NaverBot Disallow: / User-agent: grub-client Disallow: / User-agent: grub Disallow: / User-agent: WebZip Disallow: / User-agent: larbin Disallow: / User-agent: b2w/0.1 Disallow: / User-agent: Copernic Disallow: / User-agent: psbot Disallow: / User-agent: Python-urllib Disallow: / User-agent: NetMechanic Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: CherryPicker Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: WebBandit Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Crescent Disallow: / User-agent: SiteSnagger Disallow: / User-agent: ProWebWalker Disallow: / User-agent: CheeseBot Disallow: / User-agent: LNSpiderguy Disallow: / User-agent: Mozilla Disallow: / User-agent: mozilla Disallow: / User-agent: mozilla/3 Disallow: / User-agent: mozilla/4 Disallow: / User-agent: mozilla/5 Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP) Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000) Disallow: / User-agent: Alexibot Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: MIIxpc Disallow: / User-agent: Telesoft Disallow: / User-agent: Website Quester Disallow: / User-agent: moget/2.1 Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: WebStripper Disallow: / User-agent: WebSauger Disallow: / User-agent: WebCopier Disallow: / User-agent: NetAnts Disallow: / User-agent: Mister PiX Disallow: / User-agent: WebAuto Disallow: / User-agent: TheNomad Disallow: / User-agent: WWW-Collector-E Disallow: / User-agent: RMA Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: asterias Disallow: / User-agent: httplib Disallow: / User-agent: turingos Disallow: / User-agent: spanner Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) Disallow: / User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: CherryPickerElite/1.0 Disallow: / User-agent: WebBandit/3.50 Disallow: / User-agent: NICErsPRO Disallow: / User-agent: DittoSpyder Disallow: / User-agent: Foobot Disallow: / User-agent: WebmasterWorldForumBot Disallow: / User-agent: SpankBot Disallow: / User-agent: BotALot Disallow: / User-agent: lwp-trivial/1.34 Disallow: / User-agent: lwp-trivial Disallow: / User-agent: BunnySlippers Disallow: / User-agent: URLy Warning Disallow: / User-agent: Wget/1.6 Disallow: / User-agent: Wget/1.5.3 Disallow: / User-agent: Wget Disallow: / User-agent: LinkWalker Disallow: / User-agent: cosmos Disallow: / User-agent: moget Disallow: / User-agent: hloader Disallow: / User-agent: humanlinks Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Mata Hari Disallow: / User-agent: LexiBot Disallow: / User-agent: Web Image Collector Disallow: / User-agent: The Intraformant Disallow: / User-agent: True_Robot/1.0 Disallow: / User-agent: True_Robot Disallow: / User-agent: BlowFish/1.0 Disallow: / User-agent: JennyBot Disallow: / User-agent: MIIxpc/4.2 Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: WebEnhancer Disallow: / User-agent: suzuran Disallow: / User-agent: VCI WebViewer VCI WebViewer Win32 Disallow: / User-agent: VCI Disallow: / User-agent: Szukacz/1.4 Disallow: / User-agent: QueryN Metasearch Disallow: / User-agent: Openfind data gathere Disallow: / User-agent: Openfind Disallow: / User-agent: Xenu's Link Sleuth 1.1c Disallow: / User-agent: Xenu's Link Sleuth 1.2d Disallow: / User-agent: Xenu's Disallow: / User-agent: Zeus Disallow: / User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow: / User-agent: RepoMonkey Disallow: / User-agent: Openbot Disallow: / User-agent: URL Control Disallow: / User-agent: Zeus Link Scout Disallow: / User-agent: Zeus 32297 Webster Pro V2.9 Win32 Disallow: / User-agent: Webster Pro Disallow: / User-agent: EroCrawler Disallow: / User-agent: LinkScan/8.1a Unix Disallow: / User-agent: Keyword Density/0.9 Disallow: / User-agent: Kenjin Spider Disallow: / User-agent: Iron33/1.0.2 Disallow: / User-agent: Bookmark search tool Disallow: / User-agent: GetRight/4.2 Disallow: / User-agent: FairAd Client Disallow: / User-agent: Gaisbot Disallow: / User-agent: Aqua_Products Disallow: / User-agent: Radiation Retriever 1.1 Disallow: / User-agent: WebmasterWorld Extractor Disallow: / User-agent: Flaming AttackBot Disallow: / User-agent: Oracle Ultra Search Disallow: / User-agent: PerMan Disallow: / User-agent: sootle Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Haste Disallow: / User-agent: 008 Disallow: / User-agent: Yandex Disallow: / User-agent: YandexBot Disallow: / User-agent: Baiduspider Disallow: / User-agent: OmniExplorer_Bot Disallow: / User-agent: Chatalogica.com CGI Script Disallow: / User-agent: sna-0.0.1 mikeelliott@hotmail.com Disallow: / User-agent: CariBot Disallow: /
Shares / Likes / Kommentare |
---|
0 |
Es werden nur die Daten zu der angegebenen URL abgefragt und nicht zu einer eventuell vorhandenen und auf der Seite verlinkten Facebook Seite.
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)