An assortment of timestamp formats found in our (Unix) logs
When I wrote yesterday's entry on producing and parsing a common Unix log timestamp format, I expected to casually find a bunch of examples of programs using that format. What I found instead was a veritable garden of timestamp formats, and today I feel like inventorying them. My examples will use real timestamps plucked from our logs; representing them in Go or C time formatting is up to you.
(I thought about generating Go formats for all of these then realized I would need to test them all. That's too much work.)
2022-07-17 21:03:42 | Exim, rspamd, Certbot, and Apache suexec (with the time surrounded by [...]). Apt logs in /var/log/apt/* use a variant with more spaces between the date and time. |
2022-05-04 07:19:07.774 EDT | Postgresql |
2022-07-18 02:20:46.643+0000 | Libvirt QEMU logs, but these logs can also contain timestamps in another format (see later). These are in real UTC. |
2022-07-17 00:06:08 -0400 | Munge, which is used by SLURM |
[2022-05-19T13:18:25.982] | SLURM. |
2022/07/17 21:12:25.117089 | Samba (for native logs) |
Sun Jul 17 06:53:23 2022 | Clamav and Conserver. There are probably other programs using this format. |
Sun Jul 17 20:35:45.089105460 2022 | Amanda |
[17/Jul/2022:21:09:39 -0400] | The normal Apache log format, which is also used by CUPS for its logs |
17 Jul 21:45:26.022 | Redis (on Ubuntu 18.04) |
Jul 17 21:05:10 | Traditional syslog output format, including on OpenBSD machines. It's probably used by some other programs too. |
2022-07-17T21:00:02.497397-04:00 | Fedora's new syslog timestamp format;
I expect it to spread to elsewhere. This is one form of augmented RFC 3339 timestamps (with fractional seconds). |
2022-07-17T23:00:01.474Z | OpenBSD newsyslog 'the log was rolled' message
(this example is from OpenBSD 7.1). With
varying levels of precision in the decimal
seconds portion, this is also used by various
Go programs such as Prometheus, Alertmanager,
Loki, and
some but not all of libvirt's QEMU log messages. This is another form of RFC 3339 timestamps, again augmented with fractional seconds. |
2022-07-17T01:34:08-0400 | Grafana's timestamp format in its log messages in our configuration. This is a pure RFC 3339 timestamp. |
03/17/2021 | 13:37:36 | The time format for 'ipmitool sel list '
(which is really two fields, one for the
date and one for the time) |
1658107863.607 | Linux auditd timestamps in /var/log/audit/audit.log. This is seconds (and fractional seconds) since the Unix epoch. |
As they say, the great thing about standards is that there are so many of them. Some of these timestamps are based on RFC 3339, and one (SLURM's format) is close but lacks the timezone. One libvirt QEMU timestamp format seems to be GNU Date's relaxed version of RFC 3339.
Some of these timestamp formats are better than others, at least in my opinion. Timestamp formats that omit the year are unfortunate; sooner or later you'll wind up having to deduce it from context (consider a log that has messages spanning over the new year, for example).
I'm personally biased toward local time in logs (with or without an explicit offset or timezone). Other people may prefer UTC, even if the server itself isn't set to UTC as its local time zone.
Appendix: The libvirt QEMU log messages using another timestamp format
I don't know if these are supposed to be printed out or if something is off, but here's a typical one with the different timestamp format.
2022-07-18T02:22:28.273340Z qemu-system-x86_64: warning: spice: no gl-draw-done within one second
|
|