The sensible way to use Bourne shell 'here documents' in pipelines
I was recently considering a shell script where I might want to feed a Bourne shell 'here document' to a shell pipeline. This is certainly possible and years ago I wrote an entry on the rules for combining things with here documents, where I carefully wrote down how to do this and the general rule involved. This time around, I realized that I wanted to use a much simpler and more straightforward approach, one that is obviously correct and is going to be clear to everyone. Namely, putting the production of the here document in a subshell.
( cat <<EOF your here document goes here with as much as you want. EOF ) | sed | whatever
This is not as neat and nominally elegant as taking advantage of the full power of the Bourne shell's arcane rules, and it's probably not as efficient (in at least some sh implementations, you may get an extra process), but I've come around to feeling that that doesn't matter. This may be the brute force solution, but what matters is that I can look at this code and immediately follow it, and I'm going to be able to do that in six months or a year when I come back to the script.
(Here documents are already kind of confusing as it stands without adding extra strangeness.)
Of course you can put multiple things inside the
such as several here documents that you output only conditionally
(or chunks of always present static text mixed with text you have
to make more decisions about). If you want to process the entire
text you produce in some way, you might well generate it all inside
the subshell for convenience.
Perhaps you're wondering why you'd want to run a here document
through a pipe to something. The case that frequently comes up for
me is that I want to generate some text with variable substitution
but I also want the text to flow naturally with natural line lengths,
and the expansion will have variable length. Here, the natural way
out is to use
( cat <<EOF My message to $NAME goes here. It concerns $HOST, where $PROG died unexpectedly. EOF ) | fmt
fmt reflows the text regardless of how long the variables
expand out to. Depending on the text I'm generating, I may be fine
with reflowing all of it (which means that I can put all of the
text inside the subshell), or I may have some fixed formatting that
I don't want passed through
fmt (so I have to have a mix of
subshells and regular text).
Having written that out, I've just come to the obvious realization
that for simple cases I can just directly use
fmt with a here
fmt <<EOF My message to $NAME goes here. It concerns $HOST, where $PROG died unexpectedly. EOF
This doesn't work well if there's some paragraphs that I want to include only some of the time, though; then I should still be using a subshell.
(For whatever reason I apparently have a little blind spot about using here documents as direct input to programs, although there's no reason for it.)
Comments on this page:Written on 18 April 2018.