Wandering Thoughts archives


Some notes (to myself) about formatting text in jq

These days I'm having to deal with a steadily increasing number of commands that either output JSON only or where JSON is their best output option, and I want to reformat some of that JSON to a more useful or more readable text-based format. The obvious tool to do this with is jq, at least for simple reformatting (I think there's some things that are too tangled for jq). However, every time I need to do this, I keep having to look up how to format text in jq. Jq has a very big manual and a lot of features, so here's some notes to my future self about this.

In the normal case I have some fixed fields that I want to present in a line, for example information about SSH login failures:

logcli ... | jq '.[] | (.value[1], .metric.rhost, .metric.ruser)'

(I can leave off the '(' and ')' and it works, to my surprise, but I'm banging rocks here.)

First, I basically always want to use 'jq -r' so that strings aren't quoted (and strings may include what are actually numbers but rendered as strings by the tool). Then I know of several ways to produce text in a useful form.

Often the simplest way is to put the values into a JSON array and run them through the '@tsv' filter (described in the "Format strings and escaping" section of the manual), which produces tab separated output:

$ ... | jq -r '.[] | [.value[1], .metric.ruser, .metric.rhost] | @tsv'
1596   root

By itself, @tsv just puts a tab between things, which can leave me with ragged columns if the length is different enough. As various people on the Internet will tell you, the column program can be used to align the output nicely.

The next option is string interpolation:

jq -r '.[] | "\(.value[1]): \(.metric.rhost) -> \(.metric.ruser) on \(.metric.host)"'
1596: -> root on <ourhost>

String interpolation permits embedded "'s, so you can write things like '\(.metric.ruser // "<no-such-login>")' even in the interpolation string.

The third option is string concatenation, provided that all of your values are strings (or you use tostring() or @text on things).

jq -r '.[] | (.value[1] | @text) + "  " + (.metric.ruser // "??") + "@" + .metric.host + " from " + .metric.rhost'
1596  root@<ourhost> from

(I got this use of string concatenation from here.)

If I'm doing this text formatting in jq purely for output, I think it's clear that @tsv is the easiest option and has the simplest jq expression. I suspect I'd never have a reason to use string concatenation to produce the entire output line instead of doing string interpolation. Well, maybe if I'm in some shell context that makes giving jq all of those '\(' bits too hard, since string concatenation doesn't need any backslashes.

But honestly, if I need complicated formatting I'm more likely to fix jq's output up in awk with its printf. Awk printf will do a lot that's at least quite annoying in jq.

sysadmin/JqFormattingTextNotes written at 22:34:31; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.