User-agent: * Allow: / Disallow: /catalogsearch/ Disallow: /checkout/ Disallow: /*?dir* Disallow: /*?limit* User-agent: * Disallow: /in_en/catalogsearch/ User-agent: * Disallow: /catalogsearch/ User-agent: 360Spider Disallow: / User-agent: Googlebot Allow: /catalogsearch/ User-agent: Googlebot Allow: / User-agent: AwarioRssBot User-agent: AwarioSmartBot Disallow: / User-agent: coccocbot-web Disallow: / User-agent: miyao58.com Disallow: / User-agent: www.2023cp.cn Disallow: / User-agent: interact.sh Disallow: / User-agent: oast.pro Disallow: / User-agent: oast.live Disallow: / User-agent: oast.site Disallow: / User-agent: oast.online Disallow: / User-agent: oast.fun Disallow: / User-agent: oast.fun Disallow: / User-agent: oast.me Disallow: / User-agent: burpcollaborator.net Disallow: / User-agent: oastify.com Disallow: / User-agent: canarytokens.com Disallow: / User-agent: requestbin.net Disallow: / User-agent: dnslog.cn Disallow: / User-agent: Googlebot Disallow: /search/ Disallow: /*?sort= Disallow: /in_en/catalogsearch/ Disallow: /catalogsearch/ ## web Api block if no need ##Disallow:rest/default/V1/customers ##Disallow:rest/*