User-agent: * Disallow: /cgi-bin/ Disallow: /wp/ Disallow: /wordpress/ Disallow: /backup/ Disallow: /old/ Disallow: /new/ Disallow: /xmlrpc.php Disallow: /wp-login.php # Block common “SEO backlink crawlers” (many respect robots) User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: PetalBot Disallow: /