User-agent: * Disallow: /~slaven/ # Don't want automatic comments Disallow: /cgi-bin/mapserver_comment.cgi # These are quite large and unlikely to be usable for bots Disallow: /cgi-bin/bbbike-snapshot.cgi Disallow: /cgi-bin/bbbike-data.cgi # Don't crawl temporary locations (googlebot-image does this, for instance) Disallow: /mapserver/brb/tmp Disallow: /cgi-bin/mapserv?map= Disallow: /cgi-bin/mapserv?layer= Disallow: /cgi-bin/mapserver_address.cgi?coords= # Don't let bots to route searches # German Disallow: /cgi-bin/bbbike.cgi?pref_ Disallow: /cgi-bin/bbbike.cgi?start= Disallow: /cgi-bin/bbbike.cgi?startc= Disallow: /cgi-bin/bbbike.cgi?startname= Disallow: /cgi-bin/bbbike.cgi?startplz= Disallow: /cgi-bin/bbbike.cgi?startmapimg Disallow: /cgi-bin/bbbike.cgi?via= Disallow: /cgi-bin/bbbike.cgi?viamapimg Disallow: /cgi-bin/bbbike.cgi?ziel= Disallow: /cgi-bin/bbbike.cgi?ziel2= Disallow: /cgi-bin/bbbike.cgi?zielc= Disallow: /cgi-bin/bbbike.cgi?zielc_wgs84= Disallow: /cgi-bin/bbbike.cgi?zielname= Disallow: /cgi-bin/bbbike.cgi?zielplz= Disallow: /cgi-bin/bbbike.cgi?zielmapimg Disallow: /cgi-bin/bbbike.cgi?appid=android-0; Disallow: /cgi-bin/bbbike.cgi?appid=android-0& Disallow: /cgi-bin/bbbike.cgi?scope= Disallow: /cgi-bin/bbbike.cgi?scvf= # English Disallow: /cgi-bin/bbbike.en.cgi?pref_ Disallow: /cgi-bin/bbbike.en.cgi?start= Disallow: /cgi-bin/bbbike.en.cgi?startc= Disallow: /cgi-bin/bbbike.en.cgi?startname= Disallow: /cgi-bin/bbbike.en.cgi?startplz= Disallow: /cgi-bin/bbbike.en.cgi?startmapimg Disallow: /cgi-bin/bbbike.en.cgi?via= Disallow: /cgi-bin/bbbike.en.cgi?viamapimg Disallow: /cgi-bin/bbbike.en.cgi?ziel= Disallow: /cgi-bin/bbbike.en.cgi?ziel2= Disallow: /cgi-bin/bbbike.en.cgi?zielc= Disallow: /cgi-bin/bbbike.en.cgi?zielc_wgs84= Disallow: /cgi-bin/bbbike.en.cgi?zielname= Disallow: /cgi-bin/bbbike.en.cgi?zielplz= Disallow: /cgi-bin/bbbike.en.cgi?zielmapimg Disallow: /cgi-bin/bbbike.en.cgi?appid=android-0; Disallow: /cgi-bin/bbbike.en.cgi?appid=android-0& Disallow: /cgi-bin/bbbike.en.cgi?scope= Disallow: /cgi-bin/bbbike.en.cgi?scvf= # German, beta Disallow: /cgi-bin/bbbike2.cgi?pref_ Disallow: /cgi-bin/bbbike2.cgi?start= Disallow: /cgi-bin/bbbike2.cgi?startc= Disallow: /cgi-bin/bbbike2.cgi?startname= Disallow: /cgi-bin/bbbike2.cgi?startplz= Disallow: /cgi-bin/bbbike2.cgi?startmapimg Disallow: /cgi-bin/bbbike2.cgi?via= Disallow: /cgi-bin/bbbike2.cgi?viamapimg Disallow: /cgi-bin/bbbike2.cgi?ziel= Disallow: /cgi-bin/bbbike2.cgi?ziel2= Disallow: /cgi-bin/bbbike2.cgi?zielc= Disallow: /cgi-bin/bbbike2.cgi?zielc_wgs84= Disallow: /cgi-bin/bbbike2.cgi?zielname= Disallow: /cgi-bin/bbbike2.cgi?zielplz= Disallow: /cgi-bin/bbbike2.cgi?zielmapimg Disallow: /cgi-bin/bbbike2.cgi?appid=android-0; Disallow: /cgi-bin/bbbike2.cgi?appid=android-0& Disallow: /cgi-bin/bbbike2.cgi?scope= Disallow: /cgi-bin/bbbike2.cgi?scvf= # don't advertize the wap version anymore Disallow: /cgi-bin/wapbbbike.cgi Disallow: /wap # leaflet with coordsessions Disallow: /cgi-bin/bbbikeleaflet.cgi?coordssession= Disallow: /cgi-bin/bbbikeleaflet.en.cgi?coordssession= # fragezeichen pages with filled form Disallow: /BBBike/html/fragezeichenform.html?supplied_coord=;strname= # ancient googlemaps with filled form Disallow: /cgi-bin/bbbikegooglemap.cgi?wpt_or_trk= User-Agent: feedfinder Disallow: / User-Agent: msnbot Disallow: /cgi-bin/A327B109991F487BB01E1A38C1F3A941_00000 # We don't have online shops here, # and the robot generated too many 404 User-Agent: TwengaBot-Discover Disallow: / # Crawls ftp URLs badly and does not have the advertised /robot page User-Agent: SiteBot Disallow: / # No CMS here, randomly picks pages generating 404 User-Agent: CMS Crawler Disallow: / # Crawls some feed-like URLs User-Agent: Spinn3r Disallow: /feed Disallow: /atom Disallow: /rss Disallow: /index.rss Disallow: /index.rdf Disallow: /xml/rss # Comes too often (every three minutes), does not have # a proper http URL in UA, seems to be an advertising # company (oxamedia.com) User-Agent: Gnam Gnam Spider Disallow: / # Broken bot User-Agent: Pixray-Seeker Disallow: / # Broken bot, and we don't have any best buys User-agent: WBSearchBot Disallow: / ## Broken bot, but at least does not sound useless, so give it another chance... #User-agent: 008 #Disallow: / # May cause errors because of wrong encoding (and not interesting anyway) User-agent: XoviBot Disallow: / # Obviously broken behavior of the legal amazonbot User-agent: Amazonbot Disallow: /
Se han encontrado las siguientes palabras clave. Comprueba si esta página está bien optimizada para cada palabra clave en concreto.
(Deseable)