Translations:Help:Robots.txt/2/en

It determines if and when search engine web crawlers can visit a website's pages and include them in the the search engine's index.