Apache: How to turn off directory browsing
You can use either of the below commands to stop directory browsing, just add either one to your .htaccess file.
Options -Indexes |
or
Options None |
This is useful if you have a big list of files in a directory with no index.html file and you don’t want visitors to be able to browse through all of those. I’m not sure why there are two commands available for this, but either work so enjoy :).
PNR Enquiry January 11, 2015 at 11:31 pm
I want don’t to index robots.txt, sitemap.xml and some specific pages. Will you tell me, how can I do it through htaccess? Any suggestion will be appreciated; thanks in advance.
Mark McQuaid January 12, 2015 at 10:11 am
Hello There,
Thanks for the question! By default the robots.txt is what all search engines look for when querying your website. This file is what tells the search engine bots what is private and should not be indexed in their results. To restrict certain content or URL’s from being scanned and archived by these search engines you simply need to add the directories or files to the robots.txt file.
Check out these helpful links for more in-depth information.
http://kb.site5.com/bots/how-to-use-the-robots-txt-file/
https://support.google.com/webmasters/answer/6062608?hl=en
PNR Enquiry January 17, 2015 at 3:37 pm
I got the solution, just need to know whether it’s right or not. What’s your opinion about this.
Header set X-Robots-Tag “noindex”
Header set X-Robots-Tag “noindex”
James Davey January 19, 2015 at 6:38 am
That may work, yes. The best way to test this would be to add it to your .htaccess file and test the site.
PNR Status Site March 10, 2015 at 1:24 pm
I think it comes by-default to not to index robots.txt in .htacess files. By reading this I have added all noindex to my robots.txt.