2021-08-05
Using journalctl's ability to show only one service
When I wrote about sending all syslog messages to one file, one of my reasons for it was
so that I didn't have to hunt through assorted syslog files to find
out just where a program's log messages went. However, even when
you put all messages in one place, you still have to pick out the
messages you care about from all the rest. Lately, I've realized
that on systemd based Linux systems, this is working too hard and
there's a much easier way using journalctl
.
On a modern systemd based Linux system, the systemd journal knows which service unit a given message is associated with, even if the message was syslogged. This is separate from the idea of syslog facilities, which means that you don't have to care about what facility a program uses (or where your syslog configuration logs that facility).
Do you want to watch only messages from your DHCP daemon about DHCP
activity? That's easy, with no greps needed: 'journalctl -f -u
dhcpd.service
'. Well, that's on Fedora. On Ubuntu, you're probably
going to need to ask the journal to follow 'isc-dhcp-server.service'
instead. This points out one little drawback, which is that you
need to know relatively exactly what you're asking for. Journalctl
can give you all of the logs for a specific binary, like /usr/sbin/dhcpd
(with 'journalctl -f /usr/sbin/dhcpd
'; don't accidentally use
'-u
'), but as far as I know it has no convenient syntax to give
you all log messages from a given service that a particular PID is
in.
(You can do this in two steps; first use 'systemctl status <PID>
'
to get the service unit that a PID is in (along with recent log
data from the journal), then 'journalctl -u ...
' and whatever
other options. If you don't want to wait for systemd to grub around
in the journal, you can also do 'systemctl status
' and then search
out the PID. Or you can remember 'systemctl status --lines=0 <PID>
'.
I wish there was a simple systemctl option to just tell us the
service unit for a particular PID, although you can always get this
from /proc/<PID>/cgroup.)
Another thing that I should make more use of is asking for only
recent journal messages for a particular service, with '-S
'. I
originally thought that this required you to use relatively
inconvenient time stamps, but as the journalctl
and
systemd.time
manpages cover, you can use a convenient relative time syntax. If you
want the last two days of log messages for a service, this is:
journalctl -u rsyslog.service -S -2d
Now that I've read up on this, I suspect I'm going to use both -S and journalctl more than I have in the past.
(You can use this relative time syntax with both -S and --until (aka -U), which means it's relatively straightforward to narrow in on a moderate time range of interest with some basic mental math.)