Robots.txt
User-agent: *
Disallow: *?q=*&filter
Disallow: *&filterURL=*
Disallow: */search
Disallow: */cart
Disallow: */checkout
Disallow: */my-account
Disallow: */login
Disallow: *//app/etc/local.xml
Disallow: */medias/sys_master/*.gz
Disallow: */availability*
Disallow: */bom-tool-upload*
Disallow: */bom-tool*
Disallow: *undefined*
Disallow: */special-shops/*
Disallow: */shop-in-shop/*
Disallow: */Web/Downloads/*
Disallow: */shopping/*
Disallow: /compliance-document/*
Disallow: */rs-welcome
Disallow: */rs-registration
Disallow: */pf/*
# Block Bad-Bots or Useless-Bot
User-agent: CazoodleBot
User-agent: Gigabot
User-agent: admantx
User-agent: Wget
User-agent: semvisuBot
User-agent: Baiduspider
User-agent: Baiduspider-image
User-agent: BLEXBot
User-agent: YandexBot
User-agent: Yandex
User-agent: Sogou
User-agent: Exabot
User-agent: spbot
User-agent: Fetchbot
User-agent: betaBot
User-agent: LinkpadBot
User-agent: Mail.RU_Bot
User-agent: SeznamBot
User-agent: DotBot
User-agent: Yeti
User-agent: SMTBot
User-agent: Findxbot
User-agent: Genio
User-agent: BubING
User-agent: proximic
User-agent: coccoc
User-agent: GrapeshotCrawler
User-agent: savetheworldheritage.org
User-agent: ChatGPT-User
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: BrightBot
Disallow: /
Sitemap: https://www.distrelec.ch/sitemap_index.xml
(Nice to have)