User-agent: * ## Do not crawl javascript links with {{token}} Disallow: /*{{ Disallow: /*%7b%7b Disallow: /*?SID= ## Do not crawl links with ?no_cache Disallow: /*?no_cache= Disallow: /*?nocache= Disallow: /tmp/ Disallow: /vDev/ Disallow: /vPreprod/ Disallow: /webmailAPIs/ Disallow: /ctr/ Disallow: /sponsors/ Disallow: /adpics/ Disallow: /vProd/iframeSession.php Disallow: /v5/ Disallow: /v5dev/ Disallow: /chrysophylax/ Disallow: /ressources/files/ Disallow: /fr/ms/reseaupublicationsports/ Disallow: /en/ms/reseaupublicationsports/ ## Disallow some bots User-agent: Qwantify User-agent: Googlebot-Image User-agent: Amazonbot User-agent: CCBot User-agent: BLEXBot User-agent: ias_crawler User-agent: Barkrowler User-Agent: panscient.com Disallow: / User-agent: facebookexternalhit Disallow: /*stats/ ## Disallow Bad bots User-agent: PetalBot User-agent: AspiegelBot User-agent: AhrefsBot User-agent: MJ12bot User-agent: SemrushBot User-agent: Seekport User-agent: Baiduspider User-agent: 360Spider User-agent: Yisouspider User-agent: Yandex User-agent: DotBot User-agent: MauiBot User-agent: serpstatbot User-agent: Brightbot User-agent: Bytespider User-agent: Sogou web spider User-agent: Sogou inst spider User-agent: GPTBot Disallow: / User-agent: proximic Crawl-delay: 10 User-agent: bingbot Crawl-delay: 10