2022-08-10
Some notes (to myself) about formatting text in jq
These days I'm having to deal with a steadily increasing number of
commands that either output JSON only or where JSON is their best
output option, and I want to reformat some of that JSON to a more
useful or more readable text-based format. The obvious tool to do
this with is jq, at least for
simple reformatting (I think there's some things that are too
tangled for jq). However, every
time I need to do this, I keep having to look up how to format text
in jq. Jq has a very big manual and a lot of features, so
here's some notes to my future self about this.
In the normal case I have some fixed fields that I want to present in a line, for example information about SSH login failures:
logcli ... | jq '.[] | (.value[1], .metric.rhost, .metric.ruser)'
(I can leave off the '(' and ')' and it works, to my surprise, but I'm banging rocks here.)
First, I basically always want to use 'jq -r' so that strings
aren't quoted (and strings may include what are actually numbers
but rendered as strings by the tool). Then I know of several ways
to produce text in a useful form.
Often the simplest way is to put the values into a JSON array and
run them through the '@tsv' filter (described in the "Format
strings and escaping" section of the manual), which produces tab
separated output:
$ ... | jq -r '.[] | [.value[1], .metric.ruser, .metric.rhost] | @tsv' 1596 root 34.70.164.191 [...]
By itself, @tsv just puts a tab between things, which can leave me
with ragged columns if the length is different enough. As various
people on the Internet will tell you, the column program can
be used to align the output nicely.
The next option is string interpolation:
jq -r '.[] | "\(.value[1]): \(.metric.rhost) -> \(.metric.ruser) on \(.metric.host)"' 1596: 34.70.164.191 -> root on <ourhost> [...]
String interpolation permits embedded "'s, so you can write things
like '\(.metric.ruser // "<no-such-login>")' even in the interpolation
string.
The third option is string concatenation, provided that all of your
values are strings (or you use tostring() or @text on things).
jq -r '.[] | (.value[1] | @text) + " " + (.metric.ruser // "??") + "@" + .metric.host + " from " + .metric.rhost' 1596 root@<ourhost> from 34.70.164.191
(I got this use of string concatenation from here.)
If I'm doing this text formatting in jq purely for output, I think
it's clear that @tsv is the easiest option and has the simplest
jq expression. I suspect I'd never have a reason to use string
concatenation to produce the entire output line instead of doing
string interpolation. Well, maybe if I'm in some shell context that
makes giving jq all of those '\(' bits too hard, since string
concatenation doesn't need any backslashes.
But honestly, if I need complicated formatting I'm more likely to fix jq's output up in awk with its printf. Awk printf will do a lot that's at least quite annoying in jq.