How to help programmers (parts 2 and 3):
As it happens, the
is just the tip of the iceberg of Python 3's Unix problems. Here are two more ways that it helps Unix
programmers, from the release notes:
Some system APIs like
sys.argv can also present
problems when the bytes made available by the system is not
interpretable using the default encoding. Setting the
and rerunning the program is probably the best approach.
Since the release notes are not explicit, let me fill them in with
what happens in each case.
If you have environment variables with un-decodable contents, Python 3
will pretend that they don't exist (and in fact they don't as far as it
is concerned; they never made it into the
os.environ data structure).
This is worse than the
os.listdir() case, because there is no way to
work around it in your Python program; the behavior is hard-coded into
the C source of the
posix module. The only good news is that Python 3
doesn't remove these environment variables from the environment
it passes to programs it executes via things like
sys.argv, any un-decodable command line arguments (such as oddly
encoded filenames) cause your Python program to abort with a message
Could not convert argument 2 to string'. This happens whether
or not you ever import the
sys module, as it is hard coded very early
on in CPython's startup. For bonus points, the error message makes no
attempt to identify what is producing it (it doesn't even mention that
it is being produced by Python 3).
(System administrators and anyone else who deals with complex,
multi-layered systems have a special sort of affection for unidentified
As Ian Bicking noted in the comments on the
the real solution here is alternate bytes-based interfaces to both
sys.argv that (at least on Unix) would be the 'real'
versions. But that would require Python 3 admitting that Unix is not all
Unicode, which seems unlikely right now.