Dealing with my worries about Django and HTTP Basic Authentication

April 29, 2020

Last year, I attempted to upgrade our Django web app from Django 1.10 to 1.11, but ran into rather mysterious CSRF validation failures that caused me to revert back to 1.10. We have stayed there since, and the potential for this issue resurfacing has been a major blocker for moving to more recent Django versions or porting it to Python 3. I was lucky that moving from Django 1.10 to 1.11 required neither significant code changes nor a database migration, and so could be rolled back; a more major update would have left us basically marooned and having to debug Django itself, in production.

Ever since then, one of the growing focuses of my suspicions has been an interaction between our use of Apache HTTP Basic Authentication and Django's idea of a 'session' in the face of that (which apparently interacts with its CSRF protection). I rather suspect that not very many people use HTTP Basic Authentication with Django, which would allow bugs to linger here undetected for some time, and certainly Django having to magically materialize fake sessions for users authenticated this way seems like a potential source of fun problems.

Even reproducing the problem is, well, a problem, because it only manifests in a full production setup; setting up an entire duplicate of our app that runs under Django 1.11 is more than a bit tricky and I haven't done it so far. While thinking about this recently, though, it belatedly struck me that I probably don't need to go as far as a copy of our web app. If there really is a general CSRF validation issue when you're running under HTTP Basic Authentication, it should reproduce with a very basic Django app that just displays a form for you to update. I should be able to put that together fairly easily and it's easy to run it beside our web app since they have nothing to do with each other (unlike two copies of the real app, which if left alone would be trying to interact with the same things).

If I can reproduce our problems with a test app under Django 1.11, this gives me a path forward to Python 3 and a current version of Django because I can finally have confidence that we won't run into this issue in a big upgrade. Or I can find out in advance that we will run into the issue, which is much better than doing a lot of work, rolling forward, and then having it blow up badly.

(I have more thoughts about the future of our web app, but they don't fit in the margins of this entry. The short version is that we'd like to not have to do anything to our web app, but that doesn't seem very viable, cf.)

Comments on this page:

By Christopher Barts at 2020-04-29 20:33:12:

I think you have a 'can' where you intend to have a 'can't' in the first sentence of the next-to-last paragraph.

By cks at 2020-04-29 21:25:16:

If I've got the place you mean right, I do indeed mean a 'can' there. I know that the problem happens sporadically in our real application on Django 1.11. If I can reproduce it in a test application under 1.11, I can then move that test application forward to Python 3 and a current Django to see if it still happens; if it doesn't, I can have a reasonable confidence that it also won't with our real application and I can start the work to move the real application forward.

(If I can't reproduce it with a test application, we have no path forward. If I can still reproduce the problem in the updated test application under a current Django, well, at least I can file a bug report with a small test application.)

Written on 29 April 2020.
« I think you should generally be using the latest version of Go
The problem of Ubuntu 20.04, Snaps, and where your home directory is »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Apr 29 01:03:48 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.