Wandering Thoughts archives

2022-07-17

An assortment of timestamp formats found in our (Unix) logs

When I wrote yesterday's entry on producing and parsing a common Unix log timestamp format, I expected to casually find a bunch of examples of programs using that format. What I found instead was a veritable garden of timestamp formats, and today I feel like inventorying them. My examples will use real timestamps plucked from our logs; representing them in Go or C time formatting is up to you.

(I thought about generating Go formats for all of these then realized I would need to test them all. That's too much work.)

2022-07-17 21:03:42 Exim, rspamd, Certbot, and Apache suexec (with the time surrounded by [...]). Apt logs in /var/log/apt/* use a variant with more spaces between the date and time.
2022-05-04 07:19:07.774 EDT Postgresql
2022-07-18 02:20:46.643+0000 Libvirt QEMU logs, but these logs can also contain timestamps in another format (see later). These are in real UTC.
2022-07-17 00:06:08 -0400 Munge, which is used by SLURM
[2022-05-19T13:18:25.982] SLURM.
2022/07/17 21:12:25.117089 Samba (for native logs)
Sun Jul 17 06:53:23 2022 Clamav and Conserver. There are probably other programs using this format.
Sun Jul 17 20:35:45.089105460 2022 Amanda
[17/Jul/2022:21:09:39 -0400] The normal Apache log format, which is also used by CUPS for its logs
17 Jul 21:45:26.022 Redis (on Ubuntu 18.04)
Jul 17 21:05:10 Traditional syslog output format, including on OpenBSD machines. It's probably used by some other programs too.
2022-07-17T21:00:02.497397-04:00 Fedora's new syslog timestamp format; I expect it to spread to elsewhere.
This is one form of augmented RFC 3339 timestamps (with fractional seconds).
2022-07-17T23:00:01.474Z OpenBSD newsyslog 'the log was rolled' message (this example is from OpenBSD 7.1). With varying levels of precision in the decimal seconds portion, this is also used by various Go programs such as Prometheus, Alertmanager, Loki, and some but not all of libvirt's QEMU log messages.
This is another form of RFC 3339 timestamps, again augmented with fractional seconds.
2022-07-17T01:34:08-0400 Grafana's timestamp format in its log messages in our configuration. This is a pure RFC 3339 timestamp.
03/17/2021 | 13:37:36 The time format for 'ipmitool sel list' (which is really two fields, one for the date and one for the time)
1658107863.607 Linux auditd timestamps in /var/log/audit/audit.log. This is seconds (and fractional seconds) since the Unix epoch.

As they say, the great thing about standards is that there are so many of them. Some of these timestamps are based on RFC 3339, and one (SLURM's format) is close but lacks the timezone. One libvirt QEMU timestamp format seems to be GNU Date's relaxed version of RFC 3339.

Some of these timestamp formats are better than others, at least in my opinion. Timestamp formats that omit the year are unfortunate; sooner or later you'll wind up having to deduce it from context (consider a log that has messages spanning over the new year, for example).

I'm personally biased toward local time in logs (with or without an explicit offset or timezone). Other people may prefer UTC, even if the server itself isn't set to UTC as its local time zone.

Appendix: The libvirt QEMU log messages using another timestamp format

I don't know if these are supposed to be printed out or if something is off, but here's a typical one with the different timestamp format.

2022-07-18T02:22:28.273340Z qemu-system-x86_64: warning: spice: no gl-draw-done within one second
sysadmin/LogTimestampFormats written at 22:47:00; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.