How to get (or recognize) a common Unix log timestamp format in things

July 16, 2022

One common form for timestamps in Unix log files is timestamps that come out looking like '2022-07-16 20:50:35', which is to say the date (in a sensible order) and then the time, with no timezone. Unless the program writing the logs is perverse, the timestamp is in the system's local time (whatever that is), not something fixed like UTC (of course the system's local timezone may be UTC). On a casual look around our systems, this is the timestamp format used by Exim and rspamd.

Go famously has a somewhat obscure approach to formatting and parsing timestamps, which pre-defines a number of common timestamp formats. However, this one is not one of them, and for reasons beyond the scope of this entry I recently wanted to recognize timestamps in this format in Go (more or less). To save myself looking this up in the future, the Go timestamp format for this is:

2006-01-02 15:04:05

(Since this timestamp doesn't have an explicit time zone, you'll probably want to specify one somehow so that your time.Time values get the right Location.)

A number of languages print and parse timestamps using a format based on the C strftime() and strptime() functions and their formatting (although the languages may not literally be using your C library functions for this). One example is Python's time.strftime. In strftime() formatting, this timestamp is:

%Y-%m-%d %H:%M:%S

Unsurprisingly, GNU Date accepts this as a date format in case you need to produce it in shell scripts. I believe that GNU Date will also automatically recognize it as an input time format for 'date -d ...'.

(My feeling is that this is a perfectly okay timestamp format. Yes, it omits the time zone, but I feel this is a sensible thing for logs written by a program in what is most likely a fixed time zone. One of our servers is pretty much guaranteed to never change its local time zone, and that local time zone is America/Toronto.)

PS: This is far from the only timestamp format found in the log files written out by various programs that don't send things to syslog, but it's the one I needed to deal with today and I lack the energy to inventory the time strings for all of them, even just in Go.


Comments on this page:

By lilydjwg at 2022-07-17 11:43:14:

I'm always confused by this lacking of timezone, as I often access servers around the world. And I may have set my own TZ variable (because other people may prefer other timezones), so it's harder to tell which the program uses in the logs: the system one, the one in my environment, or some timezone in the program's config file?

An annoying program is PHP which ignores system timezone and default to UTC if not configured.

Another is python-telegram-bot despite in the Python world, naive datetimes often mean the local timezone from environment. What's more, it was so when I programmed my bot. So after a seemingly benign update my bot starts to mistakenly block people forever due to this and Telegram's quirks about the expiration argument. (pymongo also treats naive datetimes as UTC but I don't think it has ever changed.)

By P Kern at 2022-07-18 12:13:04:

Lately, I've been adding "day of the week" -- "%a" in strftime() -- to any log timestamps intended for "human consumption" ( %Y-%m-%d %a %H:%M:%S ) just to make it easier to match activity with typical weekly patterns.

By Phil Pirozhkov at 2022-07-24 03:48:47:

There’s a slight problem with the local time zone time, it’s when it switches to DST and back.

Written on 16 July 2022.
« 'iptables -L' doesn't show you interface matches on rules by default
An assortment of timestamp formats found in our (Unix) logs »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Jul 16 21:32:09 2022
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.