# Web Crawler configuration User-agent: bingbot Crawl-delay: 120 Disallow: /download/ Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php User-agent: DuckDuckBot User-agent: Googlebot User-agent: Slurp Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php User-agent: * Disallow: / Sitemap: https://www.parallax.com/sitemap_index.xml