== An interesting internal Django error we just got

As a result of someone trying to either exploit or damage it, [[our
account request system DjangoORMDesignPuzzleII]] just notified us
that it had hit an internal exception in the depths of Django. While
that's not too great, what was really interesting was the specific
exception and where it happened; it boiled down to:

.pn prewrap on

>  [...]
>  File ".../django/db/backends/sqlite3/base.py", line XXX, in execute
>    return Database.Cursor.execute(self, query, params)
>  OverflowError: long too big to convert

Wait, what?

It turns out that the cause of this is (to me) very interesting and also
completely explicable once we trace down the layers. We need to start at
the actual form. Among other things, this presents a <select> element
with the possible values drawn from the database. How Django implements
this in our case is that the text of each option is under our control
but the HTML form '_value_' for each option is an integer (which happens
to be the database row's internal primary key). Ie, it looks like this:

>  [...]
>  <option value="5">Blah blah</option>
>  <option value="6">More blah</option>
>  [...]

Our attacker edited the HTML (I believe using Firefox's developer
tools) to provide a really absurdly large _value_ for the option
that they picked; for example, one attempt had '518446744073709551616'
for this form element. Because [[Django is a modern web framework
DjangoSmartForms]] it of course does not simply trust this submitted
value; instead it validates it and in the process turns it into a
proper reference to an ORM object representing the particular
database row. If I'm reading the code right, this validation is
done ultimately by making a SQL query to look up the row given its
primary key (in the process this validates that it's among the set
you provided).

This is where we descend down the layers to the SQLite driver.
Because the SQLite driver is a good modern database driver, it uses
[[SQL placeholders ../programming/SQLPlaceholders]]. Since the
primary key field is an integer, this means that the SQLite driver
must convert the value passed down by Django to an actual integer
in order to pass it as a placeholder, and not just a Python integer
but an actual C-level _long_.  As it happens, Django has not passed
down the raw string value but has already called _int()_ on the
hacked up string, which has given us a Python long integer.  This
long integer is of course far too big to fit into the C-level _long_
that the SQLite driver requires and the driver notices, giving us
this _OverflowError_ (well, it turns out that it is the core Python
code that notices, but close enough).

(If you modify the form to something that is not an integer at all,
Django detects it at a much higher level and rejects the form cleanly.)

I find this an interesting error partly because of how the low level
issues involved show through. A whole cascade of things had to combine
together to create this error, including Python's unification of ints
and longs, and it is the sort of really obscure corner case that can
easily slip through and be overlooked.

(Since it can be triggered from the outside it's probably worth
reporting it as a Django bug, but I need to verify that it's still there
in the current version. We're a bit behind by now for various reasons.)

PS: We found out about this problem because one of Django's cool
features is that it can be set to email you reports about uncaught
exceptions such as this.  The reports include not just the backtrace but
also things like the form _POST_ parameters, which was vital in this
case. Without the _POST_ parameters I would have been totally lost; with
them, once I started looking the absurd values of this particular form
field jumped right out at me.