Apache directory indexes will notice and exclude blocked URLs in them
Today I learned about an interesting and nice little Apache feature.
If you have Apache generating its own automatic index pages for
filesystem directories, and you block access to some things in a
directory with <Location>
blocks, Apache's generated index won't
include what you blocked. It's as if the object doesn't exist. This
is what you want (since attempts to access those things will fail),
but it's more than I expected.
That sounds abstract, so let me make it concrete. We have an old
legacy FTP site, which we've recently
made available as a HTTPS site
because browsers are removing support for FTP. For historical reasons, this FTP site
has some symbolic links that create recursive structures; for
example, it has a public_html
symlink in the root that points
to '.
' (the current directory). Unfortunately, web spiders just
love recursive structures and will crawl through them incessantly,
with ever lengthening URLs.
(Web spider operators will probably tell you that they don't like
recursive link situations like this. I have to go by observed
behavior, which is that any number of web spiders don't appear to
notice that /public_html/
is exactly the same content as /
and /public_html/public_html/
and so on.)
We don't want to remove the symbolic links from the actual directory tree that's the FTP site, for various reasons (maybe they're there for some necessary reason, or at least have become embedded in historical FTP URLs). But the HTTPS site is new and we can drop whatever URLs we want from it. So I did the obvious simple thing:
<Location "/public_html"> Order deny,allow Deny from all </Location>
When I was verifying that this worked, I noticed that the top
level index page for the FTP site
no longer showed any public_html
entry. Testing showed that
this happened for other entries in the directory as well, if I
temporarily added them.
The mod_autoindex documentation suggests
that this is a standard feature that does general permission checks,
based on the documentation for the ShowForbidden
option to the
IndexOptions
directive. However, I haven't tested this with more complex situations,
such as <Directory> instead of <Location> or more complicated
permissions.
|
|