User-agent: baiduspider Disallow: / User-agent: Flamingo_SearchEngine+(+http://www.flamingosearch.com/bot) Disallow: / User-agent: Whitevector Crawler Disallow: / User-agent: Whitevector Disallow: / User-agent: White Vector Disallow: / User-agent: Yandex Disallow: / Crawl-delay: 220 User-agent: Slurp Crawl-delay: 10 User-agent: MJ12bot Disallow: / user-agent: AhrefsBot disallow: / User-Agent: Sogou web spider Disallow:/ # Crawlers that are kind enough to obey, but which we'd rather not have # unless they're feeding search engines. User-agent: UbiCrawler Disallow: / User-agent: DOC Disallow: / User-agent: Zao Disallow: / User-agent: Mediapartners-Google Disallow: User-Agent: * Disallow: /admin/ Disallow: /comments.php Disallow: /editprofile.php Disallow: /forgotpass.php Disallow: /register.php Disallow: /activate.php Disallow: /pm.php Disallow: /invite.php Disallow: /usersearch.php Disallow: /reportuser.php Disallow: /contact.php Disallow: /login.php Crawl-Delay: 10
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)