# # robots.txt # # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # # This file will be ignored unless it is at the root of your host: # Used: http://example.com/robots.txt # Ignored: http://example.com/site/robots.txt # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/wc/robots.html # # For syntax checking, see: # http://www.sxw.org.uk/computing/robots/check.html User-agent: eCatch User-agent: ReGet User-agent: pavuk User-agent: Grafula User-agent: BlackWidow User-agent: GetRight User-agent: WebReaper User-agent: Wget User-agent: ExtractorPro User-agent: WebLeacher User-agent: Download Demon User-agent: Internet Ninja User-agent: InterGET User-agent: Go!Zilla User-agent: ChinaClaw User-agent: RealDownload User-agent: Custo User-agent: LinkpadBot User-agent: EyeNetIE User-agent: Go-Ahead-Got-It User-agent: NetSpider User-agent: Net Vampire User-agent: GetWeb! User-agent: WebCopier User-agent: Teleport Pro User-agent: EmailSiphon User-agent: Offline Explorer User-agent: pcBrowser User-agent: Zeus User-agent: Rogerbot User-agent: WebGo IS User-agent: PageGrabber User-agent: Website eXtractor User-agent: Image Stripper User-agent: SmartDownload User-agent: Dotbot User-agent: Mass Downloader User-agent: VoidEYE User-agent: MJ12bot User-agent: SuperHTTP User-agent: LeechFTP User-agent: Website Quester User-agent: Papa Foto User-agent: GrabNet User-agent: Web Image Collector User-agent: HTTrack User-agent: NetZIP User-agent: WebSauger User-agent: JOC Web Spider User-agent: NetAnts User-agent: NearSite User-agent: SiteSnagger User-agent: Mister PiX User-agent: Express WebPictures User-agent: JetCar User-agent: Surfbot User-agent: Indy Library User-agent: Gigabot User-agent: larbin User-agent: Bot [EMAIL="[email protected]"]mailto:[email protected][/EMAIL] User-agent: EirGrabber User-agent: HMView User-agent: Xaldon WebSpider User-agent: Octopus User-agent: Web Sucker User-agent: WebAuto User-agent: Image Sucker User-agent: AhrefsBot/5.1 User-agent: WebWhacker User-agent: Widow User-agent: DISCo User-agent: SuperBot User-agent: WebFetch User-agent: AhrefsBot User-agent: Exabot User-agent: Navroad User-agent: FlashGet User-agent: MIDown tool User-agent: WebStripper User-agent: tAkeOut User-agent: Offline Navigator User-agent: SemrushBot User-agent: SemrushBot/1.1~bl User-agent: WWWOFFLE User-agent: EmailWolf User-agent: WebZIP Disallow: / User-agent: * Crawl-delay: 10 # Directories Disallow: /includes/ Disallow: /misc/ Disallow: /modules/ Disallow: /profiles/ Disallow: /scripts/ Disallow: /themes/ # Files Disallow: /CHANGELOG.txt Disallow: /cron.php Disallow: /INSTALL.mysql.txt Disallow: /INSTALL.pgsql.txt Disallow: /install.php Disallow: /INSTALL.txt Disallow: /LICENSE.txt Disallow: /MAINTAINERS.txt Disallow: /update.php Disallow: /UPGRADE.txt Disallow: /xmlrpc.php # Paths (clean URLs) Disallow: /admin/ Disallow: /comment/reply/ Disallow: /filter/tips/ Disallow: /node/add/ Disallow: /search/ Disallow: /user/register/ Disallow: /user/password/ Disallow: /user/login/ Disallow: /user/logout/ # Paths (no clean URLs) Disallow: /?q=admin/ Disallow: /?q=comment/reply/ Disallow: /?q=filter/tips/ Disallow: /?q=node/add/ Disallow: /?q=search/ Disallow: /?q=user/password/ Disallow: /?q=user/register/ Disallow: /?q=user/login/ Disallow: /?q=user/logout/
Se han encontrado las siguientes palabras clave. Comprueba si esta página está bien optimizada para cada palabra clave en concreto.
(Deseable)