2022-11-21
Using curl to test alternate (test) servers for a web site
One of the perpetual issues in system administration is that we have a new version of some web site of ours to test, for example because we're upgrading the server's operating system from Ubuntu 18.04 to 22.04. In many cases this is a problem because the web server's configuration wants to be 'example.org' but you've installed it on 'test.internal' because 'example.org' points to your current production server. Two traditional approaches to this are to modify your local /etc/hosts (or equivalent) to claim that 'example.org' has the IP address of 'test.internal', or to change the web server's Apache (or nginx or etc) configuration so that it believes it's 'test.internal' (or some suitable name) instead of 'example.org'.
As I learned today, curl has an option to support this sort of
mismatch between the server's official name and where it actually
is. Actually it
has more than one of them, but let's start with --resolve:
curl --resolve example.org:443:<IP of server> https://example.org/
As covered in the curl manual page,
the --resolve option
changes the IP address associated with a given host and port
combination. For HTTP requests, this affects both the HTTP Host
header and the TLS SNI (for HTTPS connections). You can give multiple
--resolve options if you want to, and it takes wildcards so you can do
slightly crazy things like:
curl -L --resolve *:443:<server-IP> https://example.org/redir
As mentioned by Eric Nygren, this can also be used to test an IPv6 IP before you publish it in DNS (or an alternate IPv4 IP).
Curl also has the --connect-to option, which is potentially more powerful although somewhat more verbose. It has two options that I can see, which is that it will take a host name instead of an IP address and that you can change the port (which you might need to do in order to talk to some backend server). You can wildcard everything, although in a different syntax than with --resolve, so our two examples are:
curl --connect-to example.org:443:test.internal: https://example.org/ curl -L --connect-to :443:test.internal: https://example.org/redir
You can also omit the original port, for example if you want to test HTTP to HTTPS redirection on your new test server:
curl -L --connect-to example.org::test.internal: http://example.org/
Having learned the distinction, I'll probably mostly use --connect-to because while it's slightly longer and more complicated, it's also more convenient to be able to use the test server's hostname instead of having to keep looking up its IP.
For more reading, there's Daniel Stenberg's curl another host, which also covers merely changing the Host: header.
As far as I know, curl has no option to specifically change the TLS SNI by itself, although possibly you could achieve the same effect by artfully combining --resolve (or --connect-to) and explicitly setting the Host: header. Probably there's no case where you'd want to do this (or Apache would let you do it). You can always use curl's --insecure option to ignore TLS certificate errors.