User-agent: * Crawl-delay: 6 Request-rate: 10/1 Disallow: /fileadmin/_temp_/ Disallow: /fileadmin/logs/ Disallow: /fileadmin/scripts/ Disallow: /fileadmin/templates/ Disallow: /fileadmin/users/ Disallow: /typo3/ Disallow: /typo3_src/ Disallow: /typo3conf/ Allow: /typo3conf/ext/*/Resources/Public/* Disallow: /typo3temp/ # allow Stylesheets for Google-Mobile-Check Allow: /typo3temp/*/*.css Allow: /typo3temp/*.css* Allow: /typo3temp/*.js* # Disallow Bots User-agent: SemrushBot* Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Amazonbot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Bytespider Disallow: / User-agent: meta-externalagent Disallow: / # Throttle Bots User-agent: AcademicBotRTU Crawl-delay: 60
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)