An interesting internal Django error we just got
As a result of someone trying to either exploit or damage it, our account request system just notified us that it had hit an internal exception in the depths of Django. While that's not too great, what was really interesting was the specific exception and where it happened; it boiled down to:
[...] File ".../django/db/backends/sqlite3/base.py", line XXX, in execute return Database.Cursor.execute(self, query, params) OverflowError: long too big to convert
It turns out that the cause of this is (to me) very interesting and also
completely explicable once we trace down the layers. We need to start at
the actual form. Among other things, this presents a <select> element
with the possible values drawn from the database. How Django implements
this in our case is that the text of each option is under our control
but the HTML form '
value' for each option is an integer (which happens
to be the database row's internal primary key). Ie, it looks like this:
[...] <option value="5">Blah blah</option> <option value="6">More blah</option> [...]
Our attacker edited the HTML (I believe using Firefox's developer
tools) to provide a really absurdly large
value for the option
that they picked; for example, one attempt had '518446744073709551616'
for this form element. Because Django is a modern web framework it of course does not simply trust this submitted
value; instead it validates it and in the process turns it into a
proper reference to an ORM object representing the particular
database row. If I'm reading the code right, this validation is
done ultimately by making a SQL query to look up the row given its
primary key (in the process this validates that it's among the set
This is where we descend down the layers to the SQLite driver.
Because the SQLite driver is a good modern database driver, it uses
SQL placeholders. Since the
primary key field is an integer, this means that the SQLite driver
must convert the value passed down by Django to an actual integer
in order to pass it as a placeholder, and not just a Python integer
but an actual C-level
long. As it happens, Django has not passed
down the raw string value but has already called
int() on the
hacked up string, which has given us a Python long integer. This
long integer is of course far too big to fit into the C-level
that the SQLite driver requires and the driver notices, giving us
OverflowError (well, it turns out that it is the core Python
code that notices, but close enough).
(If you modify the form to something that is not an integer at all, Django detects it at a much higher level and rejects the form cleanly.)
I find this an interesting error partly because of how the low level issues involved show through. A whole cascade of things had to combine together to create this error, including Python's unification of ints and longs, and it is the sort of really obscure corner case that can easily slip through and be overlooked.
(Since it can be triggered from the outside it's probably worth reporting it as a Django bug, but I need to verify that it's still there in the current version. We're a bit behind by now for various reasons.)
PS: We found out about this problem because one of Django's cool
features is that it can be set to email you reports about uncaught
exceptions such as this. The reports include not just the backtrace but
also things like the form
POST parameters, which was vital in this
case. Without the
POST parameters I would have been totally lost; with
them, once I started looking the absurd values of this particular form
field jumped right out at me.