Exclude folders from search engine

Support Wissensdatenbank

Du bist hier:

Exclude folder/file from search engine

Create the file robots.txt according to Wikipedia:


# robots.txt for example.com 
# I'm ruling out those webcrawlers 
User agent: Sidewinder 
Disallow: /
User agent: Microsoft.URL.Control
Disallow: /
# These directories/files should not be searched
User agent: *
Disallow: /default.html
Disallow: /Temp/ # these contents will disappear soon
Disallow: /Private/Family/Birthdays.html

With the following commands the indexing of the complete web presence is forbidden to all search engines.

User agent: *
Disallow: //

Another example: robots.txt from Wikipedia

War dieser Artikel hilfreich?
0 out Of 5 Stars
5 Stars 0%
4 Stars 0%
3 Stars 0%
2 Stars 0%
1 Stars 0%
Wie können wir diesen Artikel verbessern?
Please submit the reason for your vote so that we can improve the article.
Brauchst du Hilfe?
Related Post