Robots.txt
User-agent: *
User-agent: Baiduspider
Disallow: /
User-agent: DataForSeoBot
Disallow: /
User-agent: PetalBot
Disallow: /
User-agent: Applebot
Disallow: /
User-agent: AhrefsBot
Disallow: /
User-agent: DataForSeoBot
Disallow: /
User-agent: bingbot
Disallow: /
User-agent: bingbot/2.0
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: CheckMarkNetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)
Disallow: /
User-agent: DotBot/1.2
Disallow: /
(Nice to have)