What the 'proto' field is about in Linux 'ip route' output (and input)

September 20, 2021

Today I was looking at the output of 'ip route list' on one of our Ubuntu 18.04 machines and noticed that a few of them looked different from the others: via y.y.y.y dev eno1 via q.q.q.q dev eno1 via z.z.z.z dev eno2 proto static

The plain routes were routes to our internal "sandbox" networks and are set up by our local system to maintain them, while the route with "proto static" is an additional route added in this machine's netplan configuration. For reasons related to our solution to our problem with netplan and routes on 18.04, it would be very useful to be able to tell routes added by netplan apart from routes added by us, so I wondered if the "proto static" bit was a reliable indicator and what it actually meant.

The short summary is that this 'proto' is a theoretical route origin or routing protocol for whatever added the route, and can (in theory) be used to distinguish between different sources of routes. The routes without a "proto static" also have a routing origin associated with them, but it's the default one so ip doesn't print it by default (you can see it with 'ip -d route list'; it's 'proto boot'). So now some details.

The ip-route(8) manpage calls this field a RTPROTO and describes it this way:

[T]he routing protocol identifier of this route. RTPROTO may be a number or a string from the file /etc/iproute2/rt_protos. If the routing protocol ID is not given, ip assumes protocol boot (i.e. it assumes the route was added by someone who doesn't understand what they are doing). Several protocol values have a fixed interpretation. [...]

(The manpage doesn't explain why you should set the 'proto' to if you're adding routes through 'ip route', either manually or with scripts, or what value you should use.)

Another description of this is in the rtnetlink(7) manpage, where it's called the "route origin". The manpage mentions that these are (mostly) for user information and could be used to tag the source of routing information or distinguish between multiple routing daemons. This at least gives you some clues about why you would want to set a 'proto' and what you might use it for.

In both ip-route and rtnetlink, the 'boot' route origin is described as for routes added during boot. This is a bit misleading, because 'boot' is the default route origin (as ip-route's manpage suggests). If you use 'route add' to add routes the old fashioned way, or run 'ip route add' with no 'proto ...' option, you get a "proto boot" route. Since our route setup system did not specify a route origin, all of our routes got this default (and then 'ip route list' omitted it by default).

The 'static' route origin is described by ip-route as "the route was installed by the administrator to override dynamic routing". This seems right for a route added through static configuration such as with netplan, but it means that other things might also add routes using this route origin. For example, NetworkManager on my Fedora laptop used 'proto static' for an assortment of routes that were established when I started up OpenVPN and L2TP VPN connections, but not for WireGuard ones.

(NetworkManager also used 'proto dhcp' for the default route obtained through DHCP.)

In practice it's not Netplan making the decision about the route origin for the routes it sets up, because on our Ubuntu 18.04 setup it's really based on systemd's networkd. Netplan writes out a networkd configuration and then has networkd implement it, so it's networkd that's actually setting the 'static' route origin here. Unsurprisingly, my networkd setup on my Fedora office workstation also winds up with the default route configured as a 'proto static' route.

Given all of this, our system to set up our "sandbox" routes should likely be setting up all of its routes with a unique route origin number (as 'ip route add ... proto NNN'). This would allow us to reliably identify (and manipulate) routes created by the system, while leaving all other routes alone, whether they were set up through a system's Netplan configuration or added by hand on the fly with 'ip route add ...'.

(If I'm reading the ip-route manpage correctly, this would also allow us to remove all of these special routes at once with 'ip route flush proto NNN'.)

As far as I can tell, the Linux kernel itself makes very little use of whatever value you set for the routing origin. The rtnetlink manpage explicitly says that user created values (more or less ones other than 'kernel', 'boot', and 'static') aren't used by the kernel at all. The kernel does seem to do some checks in bridging to see if the route is or isn't a 'kernel' one, but I haven't read the code to understand what's going on.

(The normal source of 'proto kernel' routes is the direct routes you get when you configure IP addresses and networks on interfaces.)

PS: Different systems appear to have different contents for /etc/iproute2/rt_protos. If you want to add your own entries, you probably want to use drop in .conf files in /etc/iproute2/rt_protos.d/, since the iproute2 upstream changes the main file periodically.

Written on 20 September 2021.
« Microsoft's Bingbot crawler is relentless for changing pages (it seems)
Why we care about being able to (efficiently) reproduce machines »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Sep 20 22:54:30 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.