Due to discovering that one of our servers had had a network interface at 100 Mbits/sec for some time, I've become interested in what information is exposed by the Linux kernel about network interfaces in /sys, specifically in /sys/class/net/<interface>. I'm mostly interested in the information there because it's the source of what the Prometheus host agent exposes as network interface status metrics, and thus what's easy to monitor and alert on in our metrics and monitoring setup.

The overall reference for this is the Linux kernel's sysfs-class-net, which documents the /sys fields directly. For the flags sysfs file, you also need the kernel's include/uapi/linux/if.h, and for the type file, include/uapi/linux/if_arp.h. Generally sysfs-class-net is pretty straightforward about what things mean, although you may have to read several entries together. Not all interfaces have all of the files, for instance the phys_port_* files aren't present on any servers we have.

The flags file has a number of common values you may see, which I'm going to write down here for my own reference:

0x1003 or 4099 decimal

This is the common value for active Ethernet interfaces. It is MULTICAST (0x1000) plus UP (0x1) and BROADCAST (0x2). Tools like ifconfig will report RUNNING as well, but that apparently doesn't appear in sysfs.

0x1002 or 4098 decimal

This is the common value for an inactive Ethernet interface, whether or not it has a cable plugged in. It is MULTICAST plus BROADCAST, but without UP.

0x9 or 9 decimal

This is the common value for the loopback interface, made from UP (0x1) and LOOPBACK (0x8).

0x91 or 145 decimal

This is an UP (0x1), POINTOPOINT (0x10) link that is NOARP (0x80). This is the flags value of my Wireguard endpoints.

0x1091 or 4241 decimal

This is an UP (0x1), POINTOPOINT (0x10) link that is MULTICAST (0x1000) in addition to being NOARP (0x80). This is the flags value of my PPPoE DSL link's PPP connection.

The 'addr_assign_type' file is about the (Ethernet) hardware address, not any IP addresses that may be associated with the interface. A physical interface will normally have a value of 0; a value of 3 means that you specifically set the MAC address. VLAN interfaces sitting on top of physical devices have a value of 2 (they take their MAC address from the underlying devices's MAC).

The name_assign_type is somewhat random, as far as I can tell. Our Ubuntu machines all have a name assignment type value of 4 ('renamed'), while my Fedora machines mostly have a name assignment type of 3 ('named by userspace'), with one Ethernet device being a 4. My Fedora home machine's ppp0 device has a value of 1.

The most common type values are 1 (Ethernet), 772 (the loopback interface), 512 (PPP), and 65534 ('none', what my Wireguard tunnels have). Possibly someday Wireguard will have its own type value assigned in include/uapi/linux/if_arp.h.

The speed value is, as mentioned in sysfs-class-net, in Mbits/sec. The values I've seen are 100 (100M), 1000 (1G), and 10000 (10G). What gets reported for interfaces without carrier seems to depend. An UP interface with no carrier will report a speed of -1; an interface that isn't up has no speed value and attempts to read the file will report 'Invalid argument'. The Prometheus host agent turns all of these into its speed in bytes metric node_network_speed_bytes by multiplying the speed value by 125000, which normally gives you a metric value of -125000 (UP but no carrier), 12500000 (100M), 125000000 (1G), or 1250000000 (10G).

(Some Linux distributions in some situations will set additional interfaces to UP as part of trying to do DHCP on them. Otherwise they'll quietly stay down.)

The Prometheus host agent exposes what it calls 'non-numeric data' from /sys/class/net in the node_network_info metric. This gives you the device's hardware address and broadcast address, its name, its duplex (which may be blank for things that don't have a duplex mode, such as Wireguard links or virtual Ethernets), and its state (from the operstate file). Somewhat to my surprise, the operstate of the loopback interface is 'unknown', not 'up'.

Update: it turns out that the carrier file is only available for interfaces that are configured 'UP' (and then is either 0 or 1 depending on if carrier is detected). If the interface is not UP, attempting to read carrier fails with 'Invalid argument'.