For neither the first nor the last time, I've wound up in a situation where it would be quite useful for one of our machines to have what I will describe as simple symmetric routing across multiple interfaces. What I mean by this is a situation where each of the host's IP addresses is associated with an interface and when packets go out with a particular IP address, they use that interface (and the interface's default route). I call this "symmetric routing" because it makes the inbound and outbound paths the same for a given connection, which is not the case by default for a host with multiple interfaces today.

Setting this up with Linux's policy based routing is straightforward and almost mechanical. However, the setup has a lot of moving parts and there's no current automation for it that I know of. You can build your own, of course, but then that means you're stuck maintaining and operating your own automation; at that point you (we) start asking if you (we) really need symmetric routing, or if it's just a nice to have thing.

If you're directly using systemd-networkd, you can probably build something out of [Route] sections and [RoutingPolicyRule] sections, but keeping all of the sections organized for each interface and keeping the table numbers straight is up to you. Ubuntu's netplan can express similar things in its routing and routing-policy sections, but once again you're left to hand-craft everything to keep it organized (a look at the netplan examples may help get the syntax and placement of directives right). However, I'm not convinced that netplan can be made to work correctly for this because I don't see how to add direct subnet routes to tables in netplan, and direct subnet routes are required in some situations.

(It's also not always clear that you've considered all of the corner cases, especially if you're trying for a simple setup. As I've found, there can be quite a number of corner cases, some of which aren't obvious and usually appear to work.)

I don't expect the Linux kernel to have a simple configuration option or way to do this. The Linux kernel traditionally provides a bunch of low level networking options and it's up to you to build what you want out of them. But I would like things like systemd's networkd and Ubuntu's netplan to have a simple way of configuring something like this, one that reduces the amount of make-work and insures that you've covered all of the corner cases.

(I would be surprised to get it, though. It's a little bit amazing that we have policy based routing support in systemd's networkd and Ubuntu netplan.)

PS: I've historically done this in two different ways, one as isolated interfaces for testing purposes and the other as my general isolated networks on my desktop. I'm not sure which approach works better, and that sort of illustrates why I'd like to have this all handled by networkd or netplan.