# https://www.robotstxt.org/robotstxt.html # # The public pages (all in the sitemap) are for everyone. The app's own # pages say noindex in their
: /local, /login and /signup can be # crawled, so search engines read that and keep them out of results even when # someone links to them. The ones below need an account or carry a private # link (the emails' tokens), so crawlers have nothing to fetch there. User-agent: * Disallow: /premium Disallow: /subscribe Disallow: /debug Disallow: /verify-email/ Disallow: /recover-password/ Sitemap: https://wiseowl.cat/sitemap.xml