User-agent: * Crawl-delay: 10 Disallow: /wp-admin/ Disallow: /impressum/ User-agent: MJ12bot Crawl-delay: 10 Disallow: /wp-admin/ # 2015.07.28 Googlebot needs JS and CSS User-agent: Googlebot Disallow: /wp-admin/ # 2015.06.27 crawler for SentiOne User-agent: sentibot Disallow: / # 2015.04.06 SEO indexer User-agent: MegaIndex.ru/2.0 Disallow: / User-agent: MegaIndex.ru Disallow: / # 2015.02.10 AdvBot "classify web content" User-agent: AdvBot Disallow: / # 2015.01.30 XoviBot SEO bot User-agent: XoviBot Disallow: / # 2015.02.19 ??? parked domain User-agent: publiclibraryarchive.org Disallow: / # 2014.12.26. Internet Memory Research User-agent: memoryBot Disallow: / # 2014.09.26. SimilarTech, Lead Generation, Competitive Intelligence based on Web Tech Analysis User-agent: SMTBot Disallow: / # 2014.09.26. XOVI Suite, SEO & Online Marketing Tool User-agent: XoviBot Disallow: / # 2014.09.18. WebSearch User-agent: Abonti Disallow: / # 2014.09.11. The web search API User-agent: meanpathbot Disallow: / # entries without date # SEO services User-agent: SearchmetricsBot Disallow: / # panscient.com User-agent: panscient.com Disallow: / # tiscali.it search bot User-agent: IstellaBot Disallow: / # search engine User-agent: EasouSpider Disallow: / # search engine User-agent: Aboundexbot Disallow: / # Mixdata : data for big business User-agent: MixBot Disallow: / # chinese search engine User-agent: EasouSpider Disallow: / # chinese search engine User-agent: sogou spider Disallow: / # scalable, fully distributed crawler User-agent: BUbiNG Disallow: / # ??? search engine User-agent: LinkpadBot Disallow: / # search engine User-agent: Aboundexbot Disallow: / # the Internet Archive's open-source, extensible, scalable, archival-quality Web crawler User-agent: heritrix Disallow: / # kostenlose Backlinkchecker von Torsten Rückert Internetdiestleistungen User-agent: SEOkicks-Robot Disallow: / # part of Ware Bay Best Buys Search engine User-agent: WBSearchBot Disallow: / # Web crawler User-agent: ScreenerBot Disallow: / # analyses the structure of the WWW User-Agent: UnisterBot Disallow: / # search engine User-agent: SeznamBot Disallow: / # seo User-agent: SemrushBot Disallow: / # brand protection User-agent: BPImageWalker/2.0 Disallow: / # seo User-agent: lipperhey Disallow: / # seo User-agent: BLEXBot Disallow: / # search engine User-agent: wotbox Disallow: / # seo User-agent: SiteExplorer Disallow: / # plagiarism check User-agent: turnitinbot Disallow: / # search engine www.sengine.info User-agent: netEstate NE Crawler Disallow: / # news User-agent: FeedBooster Disallow: / # Apache Nutch based User-agent: Nutch Disallow: / # news portal User-agent: Mail.Ru Disallow: / # seo moz User-agent: Ezooms Disallow: / # seo User-agent: spbot Disallow: / # seo User-Agent: sistrix Disallow: / # language User-Agent: ExB Language Crawler Disallow: /
Se han encontrado las siguientes palabras clave. Comprueba si esta página está bien optimizada para cada palabra clave en concreto.
(Deseable)