Turning off HTTP basic authentication in urllib
Python's urllib module conveniently handles a great many bits of fetching URLs for you, including HTTP basic authentication. Unfortunately it does this by pausing everything to 'ask the user for the required information on the controlling terminal' (to more or less quote from the documentation). This is generally not the most useful behavior in the world, and can even be rather disconcerting.
(A more sensible default behavior would have been to either raise an exception or return a HTTP 'authentication required' status.)
As a result, all of my urllib-using programs start off by neutering this
behavior, so that if I ask them to deal with a stray URL that requires
HTTP basic authentication they'll just fail. To do this, you need to
subclass FancyURLopener and supply your own
that does nothing:
from urllib import FancyURLopenerclass MyOpener(FancyURLopener): def get_user_passwd(self, h, r, c_c = 0) return None, None
This is covered in the urllib documentation, sort of, but not even the urllib docstrings tell you what the return value should be. Apparently you are just supposed to read the source.
(Technically you can supply a do-nothing
instead, with the same effect, but I prefer to just neuter the whole
This is not the only peculiar thing urllib does. For another example, it
turns basically every problem into
IOError exceptions, generally in
blithe disregard of their standard format. (And of course it is now far
too late to fix this, because it would break backwards compatibility for
everyone who has carefully worked around this.)
There's the urllib2 module as an alternative, but the length of its documentation makes my eyes glaze over. However, on some testing it seems to be reasonably simple to use, and as a bonus it does the right thing with HTTP basic authentication. I suspect that I should start switching my code over, and certainly use it for anything new I do.
(I still use urllib because I started with Python in the 1.5 days, when there was no urllib2. Technically we still have some machines with 1.5.2, but on those machines we just use a locally built version of 2.3.4.)