Some useful features of (GNU) date for things like time conversion

April 27, 2019

As part of using shell scripts to generate Prometheus metrics, and also sometimes wanting to interact with Prometheus's API through the command line (cf), I've wound up getting deeper into the date command than I usually do.

(I've done some things with GNU date before, such as using 'date -d', including for working out Linux kernel timestamps, something which is now mostly obsolete since 'dmesg -T' will do that for you.)

If you have a timestamp of seconds since the epoch and want to convert it to a date, the GNU Date manpage itself will tell you about the '@<timestamp>' date format that it accepts as input:

date --date @2147483647

If you have something, like Prometheus' web interface, that requires input in UTC time, you can add --utc to see that instead of local time:

date -d @1556398606 --utc

By extension you can convert between local time and UTC time, for instance if you have logs in local time and need to plug their time into Prometheus in UTC. However, there is a gotcha; you may need to explicitly specify your timezone, even if it's not in the normal timestamp. So:

date -d 'Apr 26 08:24:30 EDT' --utc

To get the current time as seconds since the epoch, the format specifier is '%s':

date +%s

If you want to use timestamps from date to compute how long something took (for instance, generating metrics in a shell script), GNU date will give you nanoseconds as well with the '%N' format:

stime="$(date +%s.%N)"
etime="$(date +%s.%N)"
dur="$(echo "$etime - $stime" | bc)"

The FreeBSD version of date has most of these features if you look through its manpage and the strftime(3) manpage, but it doesn't seem to be able to output the current time in any higher precision than the second. Rough equivalences are:

date -r 2147483647
date -r 1556398606 -u
date +%s

There doesn't seem to be an easy way to convert a time string into UTC time output from what I can see (but perhaps I'm missing something). On the FreeBSD machine I have access to, the following outputs local time, not UTC time:

date -j -f '%b %d %T %Z' 'Apr 26 08:24:30 EDT' -u

You can do it in two steps, by converting the time string to a timestamp and then using 'date -r <..> -u', but that's kind of annoying. You also have to specify the time string's format instead of letting GNU date's superintelligence work it out for you (hopefully correctly). If I had to do this on a FreeBSD machine, I would write some shell scripts, and then shortly they might turn into Perl programs or something so that they'd be smarter and do everything in one step. Also, a Perl, Python, C, or Go program would definitely be required if I needed timestamps with sub-second precision.

(Or one could just compile GNU date.)

(This is the kind of entry where I write things down for my future reference, since I keep digging them out of my older scripts and sometimes taking shortcuts.)

Comments on this page:

Use `TZ=UTC date` to get zulu time in a portable way.

(It is annoying that the -d option is so unportable)

By Christopher Barts at 2019-04-27 20:51:30:

One thing I like to use date for is to make timestamps for filenames. For example, I like to download videos from YouTube, so I use youtube-dl inside a simple zsh script:

   youtube-dl --ignore-errors --all-subs -- "$@" &> $(pwd)/ytdl-$(date -Iseconds) &!

That way, I can see if there were any errors and re-download if need be.

The -Iseconds option prints the current date and time in ISO 8601 format with second resolution. It's a good compromise of sortability and readability.

By Derimagia at 2019-05-01 15:23:53:

The tools from dateutils come in handy for these type of things as well.

Written on 27 April 2019.
« Brief notes on making Prometheus instant queries with curl
A gotcha with stale metrics and *_over_time() in Prometheus »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Apr 27 17:19:49 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.