Apache's odd behavior for requests with a domain with a dot at the end

September 2, 2024

When I wrote about the fun fact that domains can end in dots and how this affects URLs, I confidently said that Wandering Thoughts (this blog) reacted to being requested through 'utcc.utoronto.ca.' (with a dot at the end) by redirecting you to the canonical form, without the final dot. Then in comments, Alex reported that they got a Apache '400 Bad Request' response when they did it. From there, things got confusing (and are still confusing).

First, this response is coming from Apache, not DWiki (the code behind the blog). You can get the same '400 Bad Request' response from https://utcc.utoronto.ca./~cks/ (a static file handled only by this host's Apache). Second, you don't always get this response; what happens depends on what you're using to access the URL. Here's what I've noticed and tested so far:

  • In some tools you'll get a TLS certificate validation failure due to a name mismatch, presumably because 'utcc.utoronto.ca.' doesn't match 'utcc.utoronto.ca'. GNU Wget2 behaves this way.

    (GNU Wget version 1.x doesn't seem to have this behavior; instead I think it may strip the final '.' off before doing much processing. My impression is that GNU Wget2 and 'GNU Wget (1.x)' are fairly different programs.)

  • on some Apache configurations, you'll get a TLS certificate validation error from everything, because Apache apparently doesn't think that that the 'dot at end' version of the host name matches any of its configured virtual host names, and so it falls back to a default TLS certificate that doesn't match what you asked for.

    (This doesn't happen with this host's Apache configuration but it does happen on some other ones I tested with.)

  • against this host's Apache, at least lynx, curl, Safari on iOS (to my surprise), and manual testing all worked, with the request reaching DWiki and DWiki then generating a redirect to the canonical hostname. By a manual test, I mean making a TLS connection to port 443 with a tool of mine and issuing:

    GET /~cks/space/blog/ HTTP/1.0
    Host: utcc.utoronto.ca.
    

    (And no other headers, although a random User-Agent doesn't seem to affect things.)

  • Firefox and I presume Chrome get the Apache '400 Bad Request' error (I don't use Chrome and I'm not going to start for this).

I've looked at the HTTP headers that Firefox's web developer tools says it's sending and they don't look particularly different or unusual. But something is getting Apache to decide this is a bad request.

(It's possible that some modern web security related headers are triggering this behavior in Apache, and only a few major browsers are sending them. I am a little bit surprised that Safari on iOS doesn't trigger this.)


Comments on this page:

By Alex Xu (Hello71) at 2024-09-03 09:29:18:

You can prove that it's not the HTTP headers by right-clicking the request in developer tools and Copy as cURL.

Firefox and Chrome send utcc.utoronto.ca. in the SNI, whereas curl and I guess your tool always normalizes it. You can test it with openssl s_client -servername, but it seems like your tool already supports SNI customization?

By cks at 2024-09-03 10:43:06:

I think something in the Go TLS stack may be stripping the final '.' on host names when handed them. This isn't as crazy as it sounds, partly because various other bits of the Go networking stack always stick the '.' on the end when doing things like looking up the hostname of the IP. So I don't think I can test this specific SNI behavior through anything written in Go, including my tool.

Written on 02 September 2024.
« The status of putting a '.' at the end of domain names
TLS Server Name Indications can be altered by helpful code »

Page tools: View Source, View Normal.
Search:
Login: Password:

Last modified: Mon Sep 2 23:16:40 2024
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.