ROBOTS.TXT

Term Main definition
ROBOTS.TXT

ROBOTS.TXT is a file that says to a search engine crawler “do not search” all or certain parts of a website that you want to keep private, therefore stopping it from appearing in search engine results.

ROBOTS.TXT is a file that says to a search engine crawler “do not search” all or certain parts of a website that you want to keep private, therefore stopping it from appearing in search engine results.

Robots.txt is a text file that resides in the root directory of a website. The file tells search engine crawlers which parts of the website they should and should not index. This can be used to keep certain pages or directories hidden from search engine results.

Hits - 73