Sitemap: https://matratzenfdm.de/sitemap.xml User-agent: * Disallow: /cgi-bin/ Disallow: /feed/ Disallow: /include/ Disallow: /login/ Disallow: /page/ Disallow: /trackback/ Disallow: /admin Disallow: /admin/user Disallow: /admin/user/login Disallow: /google+ Disallow: /ytryrty Disallow: /[email protected] Disallow: /facebook Disallow: /api.php Disallow: /cron.php Disallow: /cron.sh Disallow: /error_log Disallow: /get.php Disallow: /install.php Disallow: /*.doc$ Disallow: /*.pdf$ Disallow: /?utm_source=* Disallow: /?gclid=* # Disallow Bad Bots User-agent: Baiduspider Disallow: / User-agent: VocusBot Disallow: / User-agent: Alexibot Disallow: / User-agent: Aqua_Products Disallow: / User-agent: BackDoorBot Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: Black.Hole Disallow: / User-agent: BlackWidow Disallow: / User-agent: BlowFish Disallow: / User-agent: BlowFish/1.0 Disallow: / User-agent: BotALot Disallow: / User-agent: BotRightHere Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: Bullseye Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: BunnySlippers Disallow: / User-agent: Cegbfeieh Disallow: / User-agent: CheeseBot Disallow: / User-agent: CherryPicker Disallow: / User-agent: CherryPickerElite/1.0 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: ChinaClaw Disallow: / User-agent: Copernic Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Crescent Disallow: / User-agent: Custo Disallow: / User-agent: DISCo Disallow: / User-agent: DISCoFinder Disallow: / User-agent: DittoSpyder Disallow: / User-agent: EirGrabber Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: EroCrawler Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EyeNetIE Disallow: / User-agent: FlashGet Disallow: / User-agent: Foobot Disallow: / User-agent: FrontPage Disallow: / User-agent: Gaisbot Disallow: / User-agent: GetRight Disallow: / User-agent: GetRight/2.11 Disallow: / User-agent: GetRight/3.1 Disallow: / User-agent: GetRight/3.2 Disallow: / User-agent: GetRight/3.3 Disallow: / User-agent: GetRight/3.3.3 Disallow: / User-agent: GetRight/3.3.4 Disallow: / User-agent: GetRight/4.0.0 Disallow: / User-agent: GetRight/4.1.0 Disallow: / User-agent: GetRight/4.1.1 Disallow: / User-agent: GetRight/4.1.2 Disallow: / User-agent: GetRight/4.2 Disallow: / User-agent: GetRight/4.2c Disallow: / User-agent: GetRight/4.3 Disallow: / User-agent: GetRight/4.5 Disallow: / User-agent: GetRight/4.5a Disallow: / User-agent: GetRight/4.5b Disallow: / User-agent: GetRight/4.5b1 Disallow: / User-agent: GetRight/4.5b2 Disallow: / User-agent: GetRight/4.5b3 Disallow: / User-agent: GetRight/4.5b6 Disallow: / User-agent: GetRight/4.5b7 Disallow: / User-agent: GetRight/4.5c Disallow: / User-agent: GetRight/4.5d Disallow: / User-agent: GetRight/4.5e Disallow: / User-agent: GetRight/5.0beta1 Disallow: / User-agent: GetRight/5.0beta2 Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrabNet Disallow: / User-agent: Grafula Disallow: / User-agent: HMView Disallow: / User-agent: HTTrack Disallow: / User-agent: Harvest Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: InterGET Disallow: / User-agent: Iron33/1.0.2 Disallow: / User-agent: JennyBot Disallow: / User-agent: JetCar Disallow: / User-agent: Keyword.Density Disallow: / User-agent: LNSpiderguy Disallow: / User-agent: LeechFTP Disallow: / User-agent: LexiBot Disallow: / User-agent: LinkScan/8.1a.Unix Disallow: / User-agent: LinkWalker Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: MIIxpc Disallow: / User-agent: MIIxpc/4.2 Disallow: / User-agent: MSIECrawler Disallow: / User-agent: MJ12bot Disallow: / User-agent: MJ12bot/v1.4.0 Disallow: / User-agent: MJ12bot/v1.2.4 Disallow: / User-agent: MJ12bot/v1.2.3 Disallow: / User-agent: MJ12bot/v1.0.8 Disallow: / User-agent: MJ12bot/v1.0.7 Disallow: / User-agent: MJ12bot/v1.0.6 Disallow: / User-agent: MJ12bot/v1.0.5 Disallow: / User-agent: NICErsPRO Disallow: / User-agent: NPBot Disallow: / User-agent: NPbot Disallow: / User-agent: Navroad Disallow: / User-agent: NearSite Disallow: / User-agent: NetAnts Disallow: / User-agent: NetAnts/1.10 Disallow: / User-agent: NetAnts/1.23 Disallow: / User-agent: NetAnts/1.24 Disallow: / User-agent: NetAnts/1.25 Disallow: / User-agent: NetMechanic Disallow: / User-agent: NetSpider Disallow: / User-agent: Octopus Disallow: / User-agent: Offline.Explorer Disallow: / User-agent: Openbot Disallow: / User-agent: PageGrabber Disallow: / User-agent: PerMan Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: ProWebWalker Disallow: / User-agent: Python-urllib Disallow: / User-agent: RMA Disallow: / User-agent: ReGet Disallow: / User-agent: RealDownload Disallow: / User-agent: RealDownload/4.0.0.40 Disallow: / User-agent: RealDownload/4.0.0.41 Disallow: / User-agent: RealDownload/4.0.0.42 Disallow: / User-agent: SiteSnagger Disallow: / User-agent: SlySearch Disallow: / User-agent: SmartDownload Disallow: / User-agent: SpankBot Disallow: / User-agent: SuperBot Disallow: / User-agent: SuperHTTP Disallow: / User-agent: SuperHTTP/1.0 Disallow: / User-agent: Surfbot Disallow: / User-agent: Szukacz/1.4 Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: Telesoft Disallow: / User-agent: The.Intraformant Disallow: / User-agent: TheNomad Disallow: / User-agent: TightTwatBot Disallow: / User-agent: Titan Disallow: / User-agent: True_Robot Disallow: / User-agent: True_Robot/1.0 Disallow: / User-agent: turnitinbot Disallow: / User-agent: TurnitinBot/2.1 Disallow: / User-agent: TurnitinBot/2.0 Disallow: / User-agent: TurnitinBot/1.5 Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: URLy.Warning Disallow: / User-agent: VCI Disallow: / User-agent: VoidEYE Disallow: / User-agent: WWW-Collector-E Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Web.Image.Collector Disallow: / User-agent: WebAuto Disallow: / User-agent: WebBandit Disallow: / User-agent: WebBandit/3.50 Disallow: / User-agent: WebCopier Disallow: / User-agent: WebEnhancer Disallow: / User-agent: WebFetch Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebStripper/2.03 Disallow: / User-agent: WebStripper/2.10 Disallow: / User-agent: WebStripper/2.12 Disallow: / User-agent: WebStripper/2.13 Disallow: / User-agent: WebStripper/2.15 Disallow: / User-agent: WebStripper/2.16 Disallow: / User-agent: WebStripper/2.19 Disallow: / User-agent: WebWhacker Disallow: / User-agent: WebZIP Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: WebmasterWorldForumBot Disallow: / User-agent: Website.Quester Disallow: / User-agent: Webster.Pro Disallow: / User-agent: Wget Disallow: / User-agent: Wget/1.5.2 Disallow: / User-agent: Wget/1.5.3 Disallow: / User-agent: Wget/1.6 Disallow: / User-agent: Wget/1.7 Disallow: / User-agent: Wget/1.8 Disallow: / User-agent: Wget/1.8.1 Disallow: / User-agent: Wget/1.8.2 Disallow: / User-agent: Wget/1.9-beta Disallow: / User-agent: Widow Disallow: / User-agent: asterias Disallow: / User-agent: b2w/0.1 Disallow: / User-agent: cosmos Disallow: / User-agent: eCatch Disallow: / User-agent: eCatch/3.0 Disallow: / User-agent: hloader Disallow: / User-agent: httplib Disallow: / User-agent: humanlinks Disallow: / User-agent: ia_archiver Disallow: / User-agent: larbin Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: lwp-trivial Disallow: / User-agent: lwp-trivial/1.34 Disallow: / User-agent: moget Disallow: / User-agent: moget/2.1 Disallow: / User-agent: pavuk Disallow: / User-agent: pcBrowser Disallow: / User-agent: psbot Disallow: / User-agent: searchpreview Disallow: / User-agent: spanner Disallow: / User-agent: suzuran Disallow: / User-agent: tAkeOut Disallow: / User-agent: toCrawl/UrlDispatcher Disallow: / User-agent: turingos Disallow: / User-agent: webfetch/2.1.0 Disallow: / User-agent: wget Disallow: / User-agent: SemrushBot Disallow: / User-agent: * Disallow: /admin/ Disallow: /ajax/ User-agent: TwengaBot-2.0 Disallow:/ User-agent: TwengaBot-2.0 Disallow:/ User-agent: TwengaBot-Discover Disallow: / User-agent: TwengaBot/2.0 Disallow: / User-agent: TwengaBot Disallow: / # Begin block Bad-Robots from robots.txt User-agent: asterias Disallow:/ User-agent: BackDoorBot/1.0 Disallow:/ User-agent: Black Hole Disallow:/ User-agent: BlowFish/1.0 Disallow:/ User-agent: BotALot Disallow:/ User-agent: BuiltBotTough Disallow:/ User-agent: Bullseye/1.0 Disallow:/ User-agent: BunnySlippers Disallow:/ User-agent: Cegbfeieh Disallow:/ User-agent: CheeseBot Disallow:/ User-agent: CherryPicker Disallow:/ User-agent: CherryPickerElite/1.0 Disallow:/ User-agent: CherryPickerSE/1.0 Disallow:/ User-agent: CopyRightCheck Disallow:/ User-agent: cosmos Disallow:/ User-agent: Crescent Disallow:/ User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow:/ User-agent: DittoSpyder Disallow:/ User-agent: EmailCollector Disallow:/ User-agent: EmailSiphon Disallow:/ User-agent: EmailWolf Disallow:/ User-agent: EroCrawler Disallow:/ User-agent: ExtractorPro Disallow:/ User-agent: Foobot Disallow:/ User-agent: Harvest/1.5 Disallow:/ User-agent: hloader Disallow:/ User-agent: httplib Disallow:/ User-agent: humanlinks Disallow:/ User-agent: ia_archiver Disallow:/ User-agent: InfoNaviRobot Disallow:/ User-agent: JennyBot Disallow:/ User-agent: Kenjin Spider Disallow:/ User-agent: Keyword Density/0.9 Disallow:/ User-agent: LexiBot Disallow:/ User-agent: libWeb/clsHTTP Disallow:/ User-agent: LinkextractorPro Disallow:/ User-agent: LinkScan/8.1a Unix Disallow:/ User-agent: LinkWalker Disallow:/ User-agent: LNSpiderguy Disallow:/ User-agent: lwp-trivial Disallow:/ User-agent: lwp-trivial/1.34 Disallow:/ User-agent: Mata Hari Disallow:/ User-agent: Microsoft URL Control - 5.01.4511 Disallow:/ User-agent: Microsoft URL Control - 6.00.8169 Disallow:/ User-agent: MIIxpc Disallow:/ User-agent: MIIxpc/4.2 Disallow:/ User-agent: Mister PiX Disallow:/ User-agent: moget Disallow:/ User-agent: moget/2.1 Disallow:/ User-agent: mozilla/4 Disallow:/ User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) Disallow:/ User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 95) Disallow:/ User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 98) Disallow:/ User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT) Disallow:/ User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows XP) Disallow:/ User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows 2000) Disallow:/ User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows ME) Disallow:/ User-agent: mozilla/5 Disallow:/ User-agent: NetAnts Disallow:/ User-agent: NICErsPRO Disallow:/ User-agent: Offline Explorer Disallow:/ User-agent: Openfind Disallow:/ User-agent: Openfind data gathere Disallow:/ User-agent: ProPowerBot/2.14 Disallow:/ User-agent: ProWebWalker Disallow:/ User-agent: QueryN Metasearch Disallow:/ User-agent: RepoMonkey Disallow:/ User-agent: RepoMonkey Bait & Tackle/v1.01 Disallow:/ User-agent: RMA Disallow:/ User-agent: SiteSnagger Disallow:/ User-agent: SpankBot Disallow:/ User-agent: spanner Disallow:/ User-agent: suzuran Disallow:/ User-agent: Szukacz/1.4 Disallow:/ User-agent: Teleport Disallow:/ User-agent: TeleportPro Disallow:/ User-agent: Telesoft Disallow:/ User-agent: The Intraformant Disallow:/ User-agent: TheNomad Disallow:/ User-agent: TightTwatBot Disallow:/ User-agent: Titan Disallow:/ User-agent: toCrawl/UrlDispatcher Disallow:/ User-agent: True_Robot Disallow:/ User-agent: True_Robot/1.0 Disallow:/ User-agent: turingos Disallow:/ User-agent: URLy Warning Disallow:/ User-agent: VCI Disallow:/ User-agent: VCI WebViewer VCI WebViewer Win32 Disallow:/ User-agent: Web Image Collector Disallow:/ User-agent: WebAuto Disallow:/ User-agent: WebBandit Disallow:/ User-agent: WebBandit/3.50 Disallow:/ User-agent: WebCopier Disallow:/ User-agent: WebEnhancer Disallow:/ User-agent: WebmasterWorldForumBot Disallow:/ User-agent: WebSauger Disallow:/ User-agent: Website Quester Disallow:/ User-agent: Webster Pro Disallow:/ User-agent: WebStripper Disallow:/ User-agent: WebZip Disallow:/ User-agent: WebZip/4.0 Disallow:/ User-agent: Wget Disallow:/ User-agent: Wget/1.5.3 Disallow:/ User-agent: Wget/1.6 Disallow:/ User-agent: WWW-Collector-E Disallow:/ User-agent: Xenu's Disallow:/ User-agent: Xenu's Link Sleuth 1.1c Disallow:/ User-agent: Zeus Disallow:/ User-agent: Zeus 32297 Webster Pro V2.9 Win32 Disallow:/ # SEO-related bots User-agent: rogerbot Disallow:/ User-agent: mj12bot Disallow:/ User-agent: dotbot Disallow:/ User-agent: ahrefsbot Disallow:/ User-agent: TwengaBot-Discover Disallow: / User-agent: TwengaBot/2.0 Disallow: / User-agent: TwengaBot Disallow: / User-agent: bendercrawler Disallow: / User-agent: Bender Disallow: / User-agent: http://www.talktalk.co.uk/products/virus-alerts Disallow: / User-agent: http://www.talktalk.co.uk/products Disallow: / User-agent: http://www.talktalk.co.uk Disallow: / User-agent: Gimmie Disallow: / User-agent: Gimmie60 Disallow: / User-agent: Gimmie60/5.0 Disallow: / User-agent: domainwatcher bot Disallow: / User-agent: DomainWatcher Disallow: / User-agent: DomainWatcher Bot Disallow: / User-agent: Presto/2.7.62 Version/11.01 Disallow: / User-agent: Presto/2.7.62 Disallow: / User-agent: Presto Disallow: / User-agent: IIITBOT Disallow: / User-agent: IIITBOT/1.1 Disallow: / User-agent: [email protected] Disallow: / User-agent: Ezooms Disallow: / User-agent: Ezooms/1.0 Disallow: / User-agent: Ezooms/1.0; [email protected] Disallow: / User-agent: inet library Disallow: / User-agent: http://www.erimfatmayiseviyor.com/ Darkside of The Force Disallow: / User-agent: http://www.erimfatmayiseviyor.com/ Disallow: / User-agent: Darkside of The Force Disallow: / User-agent: The Force Disallow: / User-agent: Darkside Disallow: / User-agent: badLinks.ru`s crawler v.2 Disallow: / User-agent: badLinks.ru`s crawler Disallow: / User-agent: badLinks.ru`s Disallow: / User-agent: badLinks.ru Disallow: / User-agent: badLinks Disallow: / User-agent: librabot Disallow: / User-agent: librabot/2.0 Disallow: / User-agent: Search17Bot/1.1; http://www.search17.com/bot.php Disallow: / User-agent: Search17Bot/1.1 Disallow: / User-agent: Search17Bot Disallow: / User-agent: DoCoMo/2.0 P900i(c100;TB;W24H11)(compatible; ichiro/mobile goo;+http://help.goo.ne.jp/door/crawler.html Disallow: / User-agent: DoCoMo/2.0 P900i(c100;TB;W24H11) Disallow: / User-agent: DoCoMo/2.0 P900i Disallow: / User-agent: DoCoMo/2.0 Disallow: / User-agent: nutch Disallow: / User-agent: Nutch Disallow: / User-agent: Nutch-1.0 Disallow: / User-agent: Nutch-1.0-dev Disallow: / User-agent: agentname/Nutch-1.0-dev Disallow: / User-agent: heritrix/3.0.0 +http://www.dataclip.com Disallow: / User-agent: heritrix/3.0.0 Disallow: / User-agent: heritrix Disallow: / User-agent: Sosospider+ Disallow: / User-agent: Sosospider Disallow: / User-agent: sosospider+ Disallow: / User-agent: sosospider Disallow: / User-agent: Peew Disallow: / User-agent: Peew/1.0 Disallow: / User-agent: 123peoplebot Disallow: / User-agent: 123peoplebot/1.0 Disallow: / User-agent: pzncrawl/1.63 Disallow: / User-agent: pzncrawl Disallow: / User-agent: SiteBot Disallow: / User-agent: SiteBot/0.1 Disallow: / User-agent: Susie Disallow: / User-agent: !Susie Disallow: / User-agent: archive.org_bot Disallow: / User-agent: TCGEmail Disallow: / User-agent: http://fetch.thecyberguardian.com Disallow: / User-agent: TCGfetch Disallow: / User-agent: TCGfetch; http://fetch.thecyberguardian.com; TCGEmail Disallow: / User-agent: fetch/Nutch Disallow: / User-agent: fetch/Nutch-1.0 Disallow: / User-agent: Plukkie/1.3 Disallow: / User-agent: plukkie/1.3 Disallow: / User-agent: Plukkie Disallow: / User-agent: plukkie Disallow: / User-agent: http://www.botje.com/plukkie.htm Disallow: / User-agent: http://domino.research.ibm.com/comm/research_projects.nsf/pages/sai-crawler.callingcard.html Disallow: / User-agent: StatoolsBot Disallow: / User-agent: Domnutch-Bot/Nutch-1.0 Disallow: / User-agent: Domnutch-Bot Disallow: / User-agent: Domnutch Disallow: / User-agent: Nutch-1.0 Disallow: / User-agent: Search17Bot/1.1 Disallow: / User-agent: Search17Bot Disallow: / User-agent: SearchBot Disallow: / User-agent: SearchMonkey 1.0 Disallow: / User-agent: SearchMonkey Disallow: / User-agent: DoCoMo/2.0 P05A(c100;TB;W24H15) Disallow: / User-agent: DoCoMo/2.0 P05A Disallow: / User-agent: DoCoMo/2.0 Disallow: / User-agent: DoCoMo Disallow: / User-agent: ichiro/4.0 Disallow: / User-agent: ichiro/3.0 Disallow: / User-agent: ichiro Disallow: / User-agent: Domnutch-Bot/Nutch-1.0 Disallow: / User-agent: Domnutch-Bot Disallow: / User-agent: Domnutch Disallow: / User-agent: TwengaBot-Discover Disallow: / User-agent: TwengaBot/2.0 Disallow: / User-agent: TwengaBot Disallow: / User-agent: NetStudyBot Disallow: / User-agent: NetStudyBot1.0 Disallow: / User-agent: modspider Disallow: / User-agent: (modspider) Disallow: / User-agent: Lycos Disallow: / User-agent: Lycos_Spider Disallow: / User-agent: Lycos_Spider_ Disallow: / User-agent: Lycos_Spider_(modspider) Disallow: / User-agent: Gigabot/3.0 Disallow: / User-agent: Gigabot Disallow: / User-agent: Yeti Disallow: / User-agent: Yeti/1.0 Disallow: / User-agent: amibot Disallow: / User-agent: amibot - http://www.amidalla.de - [email protected] Disallow: / User-agent: spbot/2.0 Disallow: / User-agent: spbot Disallow: / User-agent: Mail.Ru Disallow: / User-agent: Mail.Ru/1.0 Disallow: / User-agent: YoudaoBot Disallow: / User-agent: YoudaoBot/1.0 Disallow: / User-agent: Eric Osgood/Nutch-1.0 Disallow: / User-agent: aiHitBot/1.0 Disallow: / User-agent: aiHitBot Disallow: / User-agent: 80legs Disallow: / User-agent: 80legs.com Disallow: / User-agent: http://www.80legs.com/spider.html Disallow: / User-agent: 008/0.83 Disallow: / User-agent: 008 Disallow: / User-agent: Linguee Bot Disallow: / User-agent: Linguee Disallow: / User-agent: Osgood/Nutch-1.0 Disallow: / User-agent: Osgood/Nutch Disallow: / User-agent: Osgood Disallow: / User-agent: baidu Disallow: / User-agent: Baidu Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: Baiduspider Disallow: / User-agent: DotBot/1.1 Disallow: / User-agent: DotBot Disallow: / User-agent: KaloogaBot Disallow: / User-agent: http://domino.research.ibm.com/comm/research_projects.nsf/pages/sai-crawler.callingcard.html Disallow: / User-agent: sai-crawler Disallow: / User-agent: voyager/2.0 Disallow: / User-agent: !Susie (http://www.sync2it.com/susie) Disallow: / User-agent: Susie Disallow: / User-agent: !Susie Disallow: / User-agent: eventBot Disallow: / User-agent: http://domino.research.ibm.com/comm/research_projects.nsf/pages/sai-crawler.callingcard.html Disallow: / User-agent: DKIMRepBot Disallow: / User-agent: DKIMRepBot/1.0 Disallow: / User-agent: ia_archiver-web.archive.org Disallow: / User-agent: ia_archiver-web Disallow: / User-agent: ia_archiver Disallow: / User-agent: Rankivabot Disallow: / User-agent: Rankivabot/3.2 Disallow: / User-agent: Gnomit Disallow: / User-agent: Gnomit/1.0 Disallow: / User-agent: Scooter/3.3 Disallow: / User-agent: Scooter Disallow: / User-agent: VadixBot Disallow: / User-agent: Snapbot Disallow: / User-agent: SnapPreviewBot Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: NextGenSearchBot 1 Disallow: / User-agent: twiceler Disallow: / User-agent: Twiceler Disallow: / User-agent: Twiceler-0.9 Disallow: / User-agent: cuill.com Disallow: / User-agent: panscient.com Disallow: / User-agent: panscient Disallow: / User-agent: Nutch-1.0-dev Disallow: / User-agent: Nutch-1.0 Disallow: / User-agent: Nutch Disallow: / User-agent: oBot/2.3.1 Disallow: / User-agent: oBot Disallow: / User-agent: EmailSearch Disallow: / User-agent: emailsearch Disallow: / User-agent: TurnitinBot Disallow: / User-agent: obot Disallow: / User-agent: EasyDL/3.04 Disallow: / User-agent: EasyDL Disallow: / User-agent: WebDataCentreBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: MJ12bot/v1.2.5 Disallow: / User-agent: MJ12bot/v1.3.3 Disallow: / User-agent: psbot Disallow: / User-agent: psbot/0.1 Disallow: / User-agent: CatchBot Disallow: / User-agent: catchbot Disallow: / User-agent: CatchBot/2.0 Disallow: / User-agent: heritrix/@VERSION@ Disallow: / User-agent: heritrix Disallow: / User-agent: heritrix/1.14.3.r6601+http://www.buddybuzz.net/yptrino Disallow: / User-agent: heritrix/1.14.3.r6601+http Disallow: / User-agent: heritrix/1.14.3.r6601 Disallow: / User-agent: heritrix/1.14.3 Disallow: / User-agent: discobot/1.1 Disallow: / User-agent: discobot Disallow: / User-agent: MLBot Disallow: / User-agent: mlbot Disallow: / User-agent: CazoodleBot Disallow: / User-agent: cazoodlebot Disallow: / User-agent: CazoodleBot/0.0.2 Disallow: / User-agent: HouxouCrawler/Nutch-0.9 Disallow: / User-agent: HouxouCrawleR Disallow: / User-agent: SurveyBot/2.3 Disallow: / User-agent: SurveyBot Disallow: / User-agent: surveybot/2.3 Disallow: / User-agent: DNSMine 0.9 Disallow: / User-agent: DNSMine Disallow: / User-agent: dnsmine 0.9 Disallow: / User-agent: WebDataCentreBot/1.0 Disallow: / User-agent: webdatacentreBot/1.0 Disallow: / User-agent: DBLBot/1.0 Disallow: / User-agent: DBLBot Disallow: / User-agent: WebAlta Crawler/2.0 Disallow: / User-agent: WebAlta Crawler Disallow: / User-agent: DotBot/1.0.1 Disallow: / User-agent: DotBot Disallow: / User-agent: GurujiBot/1.0 Disallow: / User-agent: MJ12bot/v1.2.3 Disallow: / User-agent: MJ12bot/v1.2.4 Disallow: / User-agent: msnbot-media/1.0 Disallow: / User-agent: msnbot-media Disallow: / User-agent: CCBot/1.0 Disallow: / User-agent: mynutchcrawler/0.8.1 Disallow: / User-agent: Gaisbot/3.0+ Disallow: / User-agent: Gaisbot/3.0 Disallow: / User-agent: Gaisbot Disallow: / User-agent: Yanga WorldSearch Bot v1.1/beta Disallow: / User-agent: Yanga WorldSearch Bot Disallow: / User-agent: Yanga Disallow: / User-agent: MSRBOT Disallow: / User-agent: Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/) Disallow: / User-agent: AurNet08 Disallow: / User-agent: Mozilla/5.0 (compatible; BuiltWith/0.1; +http://builtwith.com/bot.html) Disallow: / User-agent: BuiltWith/0.1 Disallow: / User-agent: iSpider/1.0 Disallow: / User-agent: CorenSearchBot/1.0 Disallow: / User-agent: ScoutJet Disallow: / User-agent: Exabot/3.0 Disallow: / User-agent: Exabot Disallow: / User-agent: Speedy Spider Disallow: / User-agent: AISearchBot Disallow: / User-agent: Rankivabot/3.2 Disallow: / User-agent: YebolBot Disallow: / User-agent: robotgenius Disallow: / User-agent: Atomic_Email_Hunter/4.0 Disallow: /
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)