Chris's Wiki :: blog/python/OsListdirProblem Commentshttps://utcc.utoronto.ca/~cks/space/blog/python/OsListdirProblem?atomcommentsDWiki2008-12-11T04:47:06ZRecent comments in Chris's Wiki :: blog/python/OsListdirProblem.From 64.81.145.45 on /blog/python/OsListdirProblemtag:CSpace:blog/python/OsListdirProblem:2b342073649f57a362e098a236be6b74ac0443edFrom 64.81.145.45<div class="wikitext"><p>People have talked about a similar problem with os.environ and CGI -- specifically os.environ['PATH_INFO'] is in the browser's encoding, not the "system" encoding. So if you have a CGI script running UTF8 and a system with Latin1, you'll get corrupted data from os.environ.</p>
<p>This would not be so bad, if there was a way to get bytes from os.environ -- then CGI scripts would use that, and other kinds of scripts would os.environ (or maybe a little of both, depending on the information being passed around).</p>
<p>Generally there seems to be a problem that people are mistaking bytes for text. environ['REQUEST_METHOD'] is a byte value. There are no unicode HTTP methods. But it looks like text, so people mistakenly think it should be unicode in Python 3. -- Ian Bicking</p>
</div>2008-12-11T04:47:06ZBy Chris Siebenmann on /blog/python/OsListdirProblemtag:CSpace:blog/python/OsListdirProblem:5935740085b86190615919956914d9439ff51a0cChris Siebenmann<div class="wikitext"><p>The byte-string solution cannot work everywhere either, because there are
systems that genuinely require Unicode for <code>os.listdir()</code> et al, and thus
require you to <a href="https://utcc.utoronto.ca/~cks/space/blog/python/DecodingAndEncoding"><code>.decode()</code></a> your byte-strings before
calling them with all the possibilities for failures and problems that that
implies.</p>
<p>(I believe that this implies that on such systems, <code>os.listdir()</code> also
intrinsically returns Unicode. If you require <code>os.listdir()</code> to
return byte-strings, you're back to the encoding problem again.)</p>
</div>2008-12-08T22:41:33ZFrom 12.107.56.194 on /blog/python/OsListdirProblemtag:CSpace:blog/python/OsListdirProblem:42ce1f2af88e757a06fac5448ce260b9c7c7f81fFrom 12.107.56.194<div class="wikitext"><p>"os.listdir() returns a list of bytes instances if the argument is a bytes instance" (from the same bullet point on the python what's new document you linked)</p>
<p>So while it's not perfectly portable it's not like it's a problem without a solution. In fact, one could argue that the bytes solution is the portable one as it should work everywhere.</p>
</div>2008-12-08T19:37:41ZFrom 96.237.136.153 on /blog/python/OsListdirProblemtag:CSpace:blog/python/OsListdirProblem:c6d7ee9bf6e304b222c2f0bd9165d715d6a8ff6fFrom 96.237.136.153<div class="wikitext">
<p>Yes Chris -- yes.
Making a problem worse is no way to move a language forward.</p>
<p>-- joe</p>
</div>2008-12-08T14:40:29Z