User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /search/ Disallow: *?s=* Disallow: *?*&s=* Disallow: */page/ Disallow: /explore/ Disallow: /card_preview/ User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: rogerbot Disallow:/ User-agent: dotbot Disallow:/ User-agent: AhrefsBot Disallow: / User-agent: Alexibot Disallow: / User-agent: SurveyBot Disallow: / User-agent: Xenu’s Disallow: / User-agent: Xenu’s Link Sleuth 1.1c Disallow: /