Translations:Help:Robots.txt/19/en

[At least one or more  or   entries per rule] A directory or page, relative to the root domain, that you don't want the user agent to crawl. If the rule refers to a page, it should be the full page name as shown in the browser; if it refers to a directory, it should end in a  mark.