Robots.txt
User-agent: AhrefsBot
User-agent: Amazonbot
User-agent: Baiduspider
User-agent: BLEXBot
User-agent: GrapeshotCrawler
User-agent: IonCrawl
User-agent: Jobkicks
User-agent: Linguee Bot
User-agent: Mail.RU_Bot
User-agent: MegaIndex
User-agent: MetaJobBot
User-agent: MJ12bot
User-agent: MojeekBot
User-agent: PetalBot
User-agent: Seekport
User-agent: Seekport Crawler
User-agent: SemrushBot
User-agent: SeznamBot
User-agent: Sogou web spider
User-agent: UptimeRobot
User-agent: Yandex
User-agent: YandexBot
User-agent: YandexImages
User-agent: Optimizer
Disallow: /
User-agent: *
Disallow: /ai-cache
Disallow: /cache
Disallow: /config
Disallow: /cvs
Disallow: /download
Disallow: /fehlerseiten
Disallow: /logarchiv
Disallow: /logs
Disallow: /node_modules
Disallow: /tmp
Disallow: /transfer
Disallow: /webman
Allow: /config/includes/templates
sitemap: https://www.reteo.de/sitemap/sitemap.xml
(Nice to have)