2024-09-02
Apache's odd behavior for requests with a domain with a dot at the end
When I wrote about the fun fact that domains can end in dots and how this affects URLs, I confidently said that Wandering Thoughts (this blog) reacted to being requested through 'utcc.utoronto.ca.' (with a dot at the end) by redirecting you to the canonical form, without the final dot. Then in comments, Alex reported that they got a Apache '400 Bad Request' response when they did it. From there, things got confusing (and are still confusing).
First, this response is coming from Apache, not DWiki (the code behind the blog). You can get the same '400 Bad Request' response from https://utcc.utoronto.ca./~cks/ (a static file handled only by this host's Apache). Second, you don't always get this response; what happens depends on what you're using to access the URL. Here's what I've noticed and tested so far:
- In some tools you'll get a TLS certificate validation failure due
to a name mismatch, presumably because 'utcc.utoronto.ca.' doesn't
match 'utcc.utoronto.ca'. GNU Wget2 behaves this way.
(GNU Wget version 1.x doesn't seem to have this behavior; instead I think it may strip the final '.' off before doing much processing. My impression is that GNU Wget2 and 'GNU Wget (1.x)' are fairly different programs.)
- on some Apache configurations, you'll get a TLS certificate validation
error from everything, because Apache apparently doesn't think
that that the 'dot at end' version of the host name matches any
of its configured virtual host names, and so it falls back to a
default TLS certificate that doesn't match what you asked for.
(This doesn't happen with this host's Apache configuration but it does happen on some other ones I tested with.)
- against this host's Apache, at least lynx, curl, Safari on iOS
(to my surprise), and manual testing all worked, with the request
reaching DWiki and DWiki then generating a redirect to the canonical
hostname. By a manual test, I mean making a TLS connection to
port 443 with a tool of mine
and issuing:
GET /~cks/space/blog/ HTTP/1.0 Host: utcc.utoronto.ca.
(And no other headers, although a random User-Agent doesn't seem to affect things.)
- Firefox and I presume Chrome get the Apache '400 Bad Request' error (I don't use Chrome and I'm not going to start for this).
I've looked at the HTTP headers that Firefox's web developer tools says it's sending and they don't look particularly different or unusual. But something is getting Apache to decide this is a bad request.
(It's possible that some modern web security related headers are triggering this behavior in Apache, and only a few major browsers are sending them. I am a little bit surprised that Safari on iOS doesn't trigger this.)