Wandering Thoughts archives

Using journalctl's ability to show only one service

When I wrote about sending all syslog messages to one file, one of my reasons for it was so that I didn't have to hunt through assorted syslog files to find out just where a program's log messages went. However, even when you put all messages in one place, you still have to pick out the messages you care about from all the rest. Lately, I've realized that on systemd based Linux systems, this is working too hard and there's a much easier way using journalctl.

On a modern systemd based Linux system, the systemd journal knows which service unit a given message is associated with, even if the message was syslogged. This is separate from the idea of syslog facilities, which means that you don't have to care about what facility a program uses (or where your syslog configuration logs that facility).

Do you want to watch only messages from your DHCP daemon about DHCP activity? That's easy, with no greps needed: 'journalctl -f -u dhcpd.service'. Well, that's on Fedora. On Ubuntu, you're probably going to need to ask the journal to follow 'isc-dhcp-server.service' instead. This points out one little drawback, which is that you need to know relatively exactly what you're asking for. Journalctl can give you all of the logs for a specific binary, like /usr/sbin/dhcpd (with 'journalctl -f /usr/sbin/dhcpd'; don't accidentally use '-u'), but as far as I know it has no convenient syntax to give you all log messages from a given service that a particular PID is in.

(You can do this in two steps; first use 'systemctl status <PID>' to get the service unit that a PID is in (along with recent log data from the journal), then 'journalctl -u ...' and whatever other options. If you don't want to wait for systemd to grub around in the journal, you can also do 'systemctl status' and then search out the PID. Or you can remember 'systemctl status --lines=0 <PID>'. I wish there was a simple systemctl option to just tell us the service unit for a particular PID, although you can always get this from /proc/<PID>/cgroup.)

Another thing that I should make more use of is asking for only recent journal messages for a particular service, with '-S'. I originally thought that this required you to use relatively inconvenient time stamps, but as the journalctl and systemd.time manpages cover, you can use a convenient relative time syntax. If you want the last two days of log messages for a service, this is:

journalctl -u rsyslog.service -S -2d

Now that I've read up on this, I suspect I'm going to use both -S and journalctl more than I have in the past.

(You can use this relative time syntax with both -S and --until (aka -U), which means it's relatively straightforward to narrow in on a moderate time range of interest with some basic mental math.)