# The Facebook Crawler User-agent: Facebot Allow: / User-agent: sogou spider Disallow: / User-agent: Baiduspider Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: Flamingo_SearchEngine Disallow: / User-agent: WebReaper Disallow: / User-agent: Search Atlas Disallow: / User-agent: SearchAtlas.com Disallow: / User-agent: Pinterestbot Disallow: / User-agent: Slurp Crawl-delay: 60 User-agent: * Disallow: /typo3temp/* Allow: /typo3temp/*.css Allow: /typo3temp/*.css.*.gzip Allow: /typo3temp/*.js Allow: /typo3temp/*.js.*.gzip Allow: /typo3temp/*.jpg Allow: /typo3temp/*.gif Allow: /typo3temp/*.png # Only allow URLs generated with RealURL #Disallow: /*?id=* #Disallow: /*&id=* # L=0 is the default language Disallow: /*?L=0* Disallow: /*&L=0* # typeNum = 98 is usually the print version. Disallow: /*?type=98* Disallow: /*&type=98* # Should always be protected (.htaccess) Disallow: /*/Private/* Disallow: /fileadmin/templates/html/* Disallow: /*/Configuration/* Disallow: /fileadmin/user_upload/presse/* Disallow: *.sql Disallow: *.sql.gz Disallow: /typo3conf/*
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)