How to see raw USB events on Linux via usbmon

June 11, 2017

Suppose, not entirely hypothetically, that you're wondering if your USB keyboard actually does anything for certain key presses, such as its Fn button plus a normal letter key like 's'. One way to try to find out is just to type the key combination while sitting at your shell prompt or in your editor or the like, and see what happens. A more elaborate method is to fire up xev and see what it says about X events, since there are a number of things that can happen between the level of X events and what your shell sees. Of course this starts to hint at the broad problem, which is that a modern graphical Linux environment has all sorts of layers that may swallow or distort raw events, so not seeing anything in the shell or in xev only means that things didn't make it that far.

(This is useful knowledge, of course, especially if your ultimate goal is getting characters to your shell or to an X program that should recognize them. And hitting that key produces some event, xev will tell you what it's been turned into.)

Usefully, the Linux kernel gives us a way to bypass all of the (potential) layers of input processing and see the raw USB events being generated (or not generated, as the case may be). Looking at the presence or absence of raw events is pretty definite. If the keyboard is not generating any USB events when you press some keys, well, that's it. How you do this is with the kernel's usbmon system, as covered in the kernel's usbmon.txt and Ubuntu's page on debugging USB. For quick checks, the text interface in /sys/kernel/debug/usb/usbmon is your most convenient option, but Linux distributions seem to vary as to whether you need to load a usbmon kernel module or not to get it.

(On my Fedora 25 machines, everything is ready to go out of the box, with no kernel modules needing to be loaded; the Fedora kernels are apparently built with CONFIG_USB_MON=y. On a relatively stock Ubuntu 16.04 server, there's a usbmon module and it's not loaded by default.)

Wireshark can capture and interpret USB bus traffic, per here, and in theory should be a good way to see the details of USB events in a more user-friendly format than the kernel's text dump. In practice I can't seem to persuade the Fedora 25 version to give me useful information here. I find it more helpful to read the output from the text interface, which at least lets me distinguish one sort of event from another (for example, the mouse scrollwheel going in one direction versus the other direction). Possibly I'm missing some magic Wireshark options, especially since I don't use Wireshark very often. Alternately, I'd need to know a non-casual amount about USB message formats and the details of the USB protocol in order to understand what Wireshark is showing me and extract the things I'm interested in.

(There also may be an issue that apparently Wireshark may only do a good job decoding things if it sees you plug in the USB device. This is perhaps sensible behavior for Wireshark, or even necessary, but it's not very useful for checking the details of what my (only) keyboard or mouse generate. I'm not really enthused about unplugging and then replugging them; it has somewhat annoying side effects.)

As a side note, since you can only monitor an entire USB buss (or all busses at once), it's not necessarily very important to be detailed about identifying what USB device is where. For instance, on my home machine, the only USB bus that reports any non-hub devices is bus 2, so even if a plain 'lsusb' doesn't clearly identify the specific USB device that's my keyboard or mouse, it's a pretty good bet that they're on bus 2. 'lsusb -t' can also give strong hints; on my home machine, all 'Class=Human Interface Device' USB devices are on bus 2 even if lsusb doesn't tell me exactly what their (product) names are.

(I started writing this entry just to have some information about the usbmon feature recorded for my later use, and wound up learning useful things about Wireshark's USB stuff.)

PS: Because Wireshark is actually using libpcap to get the USB traffic from the kernel, you can also use tcpdump to capture USB traces, for example 'tcpdump -i usbmon2 -s 0 -w /tmp/usbmon.pcap'. This may be handy if you have a server seeing USB oddities; at least around here, we're far more likely to have basic tcpdump installed on random machines than we are to put Wireshark on them. As usual, you can use tcpdump to capture the packet trace, transfer it to your workstation, and run Wireshark on the capture to decode and analyze it and so on.

(This is also how I avoid running Wireshark as root even if I'm capturing the traffic on my own machine.)

Comments on this page:

By Pete Zaitcev at 2017-06-12 14:24:22:

Ironically, usbmon is shaping to be the most durable of my kernel contributions. Nobody uses floppies anymore. What's left of sparc code is going to die with sparc. ALSA superceded ymfpci. The ub was evicted (for a good reason). But usbmon is still around despite being a day hack.

Fedora is correct at keeping usbmon on. It's intended to be an example of "always on" tracing facility. Since I was at Red Hat at the time, it was difficult or impossible to tell people to run custom debug kernels, not to mention the effort required to build them. It was much more productive to have kernels with lightweight diagnostics that can be counted upon.

Written on 11 June 2017.
« One downside of the traditional style of writing Unix manpages
Why filing away mailing lists for a while has improved my life »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jun 11 00:52:36 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.