Disallow: /*searchTerm= Disallow: /global/wGlobal/scripts/ # # Google Image Crawler Setup - having crawler-specific sections makes it ignore generic e.g * User-agent: Googlebot-Image Allow: / #User-agent: Googlebot #Disallow: User-agent: Googlebot Allow: / # # Yandex tends to be rather aggressive, may be worth keeping them at arms lenght User-agent: YandexBot Crawl-delay: 20 User-agent: Pinterest Crawl-delay: 1 # # Crawlers Setup #User-agent: * # Block Ahrefs User-agent: AhrefsBot Disallow: / # Block SEOkicks User-agent: SEOkicks-Robot Disallow: / # Block SISTRIX User-agent: SISTRIX Crawler Disallow: / # Block Uptime robot User-agent: UptimeRobot/2.0 Disallow: / User-agent: 008 Disallow: / # Block Ezooms Robot User-agent: Ezooms Robot Disallow: / # Block Perl LWP User-agent: Perl LWP Disallow: / User-agent: BLEXBot Disallow: / User-agent: SEOkicks Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: MajesticSEO Disallow: / User-agent: BacklinkCrawler Disallow: / User-agent: xovi Disallow: / User-agent: XoviBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: spbot Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: search17 Disallow: / User-agent: AhrefsBot Disallow: / User-agent: ia_archiver Disallow: / User-agent: TurnitinBot Disallow: / User-agent: SlySearch Disallow: / User-agent: findlinks Disallow: / User-agent: magpie-crawler Disallow: / User-agent: Pixray-Seeker Disallow: / User-agent: 008 Disallow: / User-agent: Ezooms Disallow: / User-agent: lb-spider Disallow: / User-agent: WBSearchBot Disallow: / User-agent: psbot Disallow: / User-agent: HuaweiSymantecSpider Disallow: / User-agent: EC2LinkFinder Disallow: / User-agent: htdig Disallow: / User-agent: SemrushBot Disallow: / User-agent: discobot Disallow: / User-agent: linkdex.com Disallow: / User-agent: SeznamBot Disallow: / User-agent: EdisterBot Disallow: / User-agent: SWEBot Disallow: / User-agent: picmole Disallow: / User-agent: Yeti Disallow: / User-agent: Yeti-Mobile Disallow: / User-agent: PagePeeker Disallow: / User-agent: CatchBot Disallow: / User-agent: yacybot Disallow: / User-agent: netEstateNECrawler Disallow: / User-agent: SurveyBot Disallow: / User-agent: COMODOSSLChecker Disallow: / User-agent: Comodo-Certificates-Spider Disallow: / User-agent: gonzo Disallow: / User-agent: schrein Disallow: / User-agent: AfiliasWebMiningTool Disallow: / User-agent: suggybot Disallow: / User-agent: bdbrandprotect Disallow: / User-agent: BPImageWalker Disallow: / User-agent: Updownerbot Disallow: / User-agent: lex Disallow: / User-agent: ContentCrawler Disallow: / User-agent: DCPbot Disallow: / User-agent: KaloogaBot Disallow: / User-agent: MLBot Disallow: / User-agent: iCjobs Disallow: / User-agent: oBot Disallow: / User-agent: WebmasterCoffee Disallow: / User-agent: Qualidator Disallow: / User-agent: Webinator Disallow: / User-agent: Scooter Disallow: / User-agent: thunderstone Disallow: / User-agent: larbin Disallow: / User-agent: OpidooBOT Disallow: / User-agent: ips-agent Disallow: / User-agent: TinEye Disallow: / User-agent: UnisterBot Disallow: / User-agent: Unister Disallow: / User-agent: ReverseGet Disallow: / User-agent: DotBot Disallow: / User-agent: * Allow: /
Se han encontrado las siguientes palabras clave. Comprueba si esta página está bien optimizada para cada palabra clave en concreto.
(Deseable)