I don't understand Linux iptables NAT as well as I should
I'm currently having a problem with my DSL link where after a restart of the link (such as a power outage), I now can't reach any number of networks over it; unfortunately, quite a lot of these networks are major hosting providers like AWS, Cloudflare, OVH, Google, and so on. Fortunately I can reach the other side of my IPSec tunnel and the other side of my IPSec tunnel can reach all of these networks. For the most part all I want to do that involves these networks is visit web sites hosted on them, so after manually adding routes to force traffic over my IPSec tunnel got tiring I thought that I'd use iptables to just redirect all outgoing HTTP and HTTPS traffic over my IPSec tunnel, as I've done before.
So I set up all of the necessary
iptables rules (or what I thought
were the necessary rules), I could see my HTTP requests flowing out
the right interface with the right address, and nothing actually
worked. Whoops. Never mind my browsers, I couldn't even get a simple
TCP-level connection to, well, connect. Tcpdump said that the return
traffic was coming back but nothing was doing anything with it. At
first I thought this was a MTU issue, because unlike the last
time I did this, I'm forcing traffic
that was originally going out an interface with a higher MTU into
one with a smaller one. Some magic iptables invocations to fix this
later I realized that it can't possibly be the problem if a plain
TCP connection fails to complete.
I will cut to the chase: I've failed to figure out what the problem
is with my iptables rules (and my setup). I tried a number of things
which I thought might be the problem (turning on forwarding for the
IPSec tunnel, for instance, in case the kernel was dropping packets
for that reason, and fiddling with the rp_filter setting), and they all failed. I tried directions on
turning on packet tracing for iptables debugging and they
produced nothing. I looked in
/proc/net/nf_conntrack, which was
kind of educational but had no enlightenment for me; the best I
could see was that my connections never transitioned from
(Other NAT activity was and is fine. Well. Relatively fine. It gets NAT'd properly when it can reach things at all.)
All of this is frustrating, but it shows up a larger problem: I don't really know my way around NAT in iptables, and to some extent I don't know my way around iptables at all. Iptables is quite complicated, especially once you add in NAT magic, and while I once dug into it to some extent (cf), I've let that knowledge fade. At one point this was reasonably okay because I was only doing relatively simple things with iptables. Those days are sort of over now and my relative lack of understanding is getting in the way. I need to figure out how to get back up to speed here, even if I know I'm going to yearn for the relative simplicity of OpenBSD's PF.
(Unfortunately it's difficult to feel motivated to dig into NAT right now when the whole DSL routing situation makes me want to set things on fire.)
PS: My views of Linux iptables versus OpenBSD PF are complicated. I will summarize by saying that I have wanted to set both of them on fire at different times.