# robots.txt for https://startit-x.com/en/ sitemap: https://startit-x.com/en/sitemaps-1-sitemap.xml # live - don't allow web crawlers to index cpresources/ or vendor/ User-agent: * Crawl-delay: 20 Disallow: /cpresources/ Disallow: /vendor/ Disallow: /.env Disallow: /cache/