Files
website/_site/robots.txt

9 lines
161 B
Plaintext

User-agent: ia_archiver
Disallow: /
User-agent:*
Sitemap: http://localhost:4000/sitemap.xml
Disallow: /assets/*
Disallow: /assets/*.jpg
Disallow: /assets/*.png