Esta página no es accesible para el bot de Seobility porque está excluido en el archivo Robots.txt. El bot de Seobility no rastrea nada que excluya a Seobility o *.
URL problemática: http://dot.ny.gov
# $Id: robots.txt # # This is a file retrieved by webwalkers a.k.a. spiders that # conform to a defacto standard. # See <URL:http://www.robotstxt.org/wc/exclusion.html#robotstxt> # # Format is: # User-agent: <name of spider> # Disallow: <nothing> | <path> # ----------------------------------------------------------------------------- User-agent: nys-crawler Disallow: User-agent: WC3-checklink Disallow: User-agent: Googlebot Disallow: User-agent: Inktomi Slurp Disallow: User-agent: MSNBot Disallow: User-agent: AskJeeves Disallow: User-agent: InfoSeek Robot 1.0 Disallow: User-agent: InfoSeek Sidewinder Disallow: User-agent: * Disallow: / Disallow: // Disallow: /portal/page/portal/ Disallow: /pls/ Disallow: /portalHelp2/ Disallow: /portal/pls/ Disallow: /tmp