Some notes on what's in Linux's
/sys/class/net for network interface status
Due to discovering that one of our servers had had a network
interface at 100 Mbits/sec for some time,
I've become interested in what information is exposed by the Linux
kernel about network interfaces in
/sys, specifically in
/sys/class/net/<interface>. I'm mostly interested in the information
there because it's the source of what the Prometheus host agent exposes as network
interface status metrics, and thus what's easy to monitor and alert
on in our metrics and monitoring setup.
The overall reference for this is the Linux kernel's sysfs-class-net,
which documents the
/sys fields directly. For the
file, you also need the kernel's include/uapi/linux/if.h,
and for the
type file, include/uapi/linux/if_arp.h.
Generally sysfs-class-net is pretty straightforward about what
things mean, although you may have to read several entries together.
Not all interfaces have all of the files, for instance the
files aren't present on any servers we have.
flags file has a number of common values you may see, which I'm
going to write down here for my own reference:
- 0x1003 or 4099 decimal
- This is the common value for active Ethernet
interfaces. It is MULTICAST (0x1000) plus UP (0x1) and BROADCAST (0x2).
ifconfigwill report RUNNING as well, but that apparently doesn't appear in sysfs.
- 0x1002 or 4098 decimal
- This is the common value for an inactive
Ethernet interface, whether or not it has a cable plugged in. It
is MULTICAST plus BROADCAST, but without UP.
- 0x9 or 9 decimal
- This is the common value for the loopback interface,
made from UP (0x1) and LOOPBACK (0x8).
- 0x91 or 145 decimal
- This is an UP (0x1), POINTOPOINT (0x10) link that
is NOARP (0x80). This is the
flagsvalue of my Wireguard endpoints.
- 0x1091 or 4241 decimal
- This is an UP (0x1), POINTOPOINT (0x10) link
that is MULTICAST (0x1000) in addition to being NOARP (0x80). This is
flagsvalue of my PPPoE DSL link's PPP connection.
addr_assign_type' file is about the (Ethernet) hardware
address, not any IP addresses that may be associated with the
interface. A physical interface will normally have a value of 0; a
value of 3 means that you specifically set the MAC address. VLAN
interfaces sitting on top of physical devices have a value of 2
(they take their MAC address from the underlying devices's MAC).
name_assign_type is somewhat random, as far as I can tell.
Our Ubuntu machines all have a name assignment type value of 4
('renamed'), while my Fedora machines mostly have a name assignment
type of 3 ('named by userspace'), with one Ethernet device being a
4. My Fedora home machine's
ppp0 device has a value of 1.
The most common
type values are 1 (Ethernet), 772 (the loopback
interface), 512 (PPP), and 65534 ('none', what my Wireguard tunnels
have). Possibly someday Wireguard will have its own
assigned in include/uapi/linux/if_arp.h.
speed value is, as mentioned in sysfs-class-net, in
Mbits/sec. The values I've seen are 100 (100M), 1000 (1G), and 10000
(10G). What gets reported for interfaces without carrier seems to
depend. An UP interface with no carrier will report a speed of -1;
an interface that isn't up has no
speed value and attempts to
read the file will report '
Invalid argument'. The Prometheus host
agent turns all of these into its speed in bytes metric
node_network_speed_bytes by multiplying the speed value by
125000, which normally gives you a metric value of -125000 (UP but
no carrier), 12500000 (100M), 125000000 (1G), or 1250000000 (10G).
(Some Linux distributions in some situations will set additional interfaces to UP as part of trying to do DHCP on them. Otherwise they'll quietly stay down.)
The Prometheus host agent exposes what it calls 'non-numeric data'
/sys/class/net in the
node_network_info metric. This
gives you the device's hardware address and broadcast address,
its name, its duplex (which may be blank for things that don't have
a duplex mode, such as Wireguard links or virtual Ethernets), and
its state (from the
operstate file). Somewhat to my surprise, the
operstate of the loopback interface is 'unknown', not 'up'.
Update: it turns out that the
carrier file is only available for
interfaces that are configured 'UP' (and then is either 0 or 1
depending on if carrier is detected). If the interface is not UP,
attempting to read
carrier fails with 'Invalid argument'.