Skip to content

Commit

Permalink
Update the 'robots.txt' file.
Browse files Browse the repository at this point in the history
  • Loading branch information
@frozar
frozar committed Oct 7, 2021
1 parent 44c09a8 commit 18ea647
Showing 1 changed file with 10 additions and 1 deletion.
11 changes: 10 additions & 1 deletion public/robots.txt
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
# https://www.robotstxt.org/robotstxt.html
# User-agent: *
# Disallow:

# https://nextjs.org/learn/seo/crawling-and-indexing/robots-txt
# Block all crawlers for /api
User-agent: *
Disallow:
Disallow: /api

# Allow all crawlers
User-agent: *
Allow: /

1 comment on commit 18ea647

@vercel
Copy link

@vercel vercel bot commented on 18ea647 Oct 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Successfully deployed to the following URLs:

Please sign in to comment.