== (Ab)using _awk_ on the fly Suppose that you have a file with lines of the form: .pn prewrap on > host1: package1 package2 package3 > host2: package3 package5 > ahost3: package1 package2 package3 You want to transform this into something that looks like: > package1 package2 package3 > host1 ahost3 > package3 package5 > host2 In other words, aggregate together all of the hosts with a common set of packages (in this case, packages to update that require manual work). One of the problems of modern Unix is that there are simply too many programs that do random chunks of text processing for anyone except a specialist to remember or even know them all and know what they do. Thus it's quite possible that there any number of clever ways to do this with relatively standard and widely available GNU or other tools. I just don't know what they are off the top of my head and it is much faster to use tools that I know, even in brute force ways, than to go searching and searching and maybe not find anything. So here is how I did this, on the fly, using tools that I'm already familiar with (which primarily means _awk_). Let's assume the file is _pkglist_: > sort -b -k2 pkglist | sed 's/: */:/' | > awk -F: '$2 == last {sum = sum " " $1} > $2 != last && last {printf "%s\n\t%s\n", last, sum} > $2 != last {sum = $1; last = $2} > END {printf "%s\n\t%s\n", last, sum}' (The actual version I used put all of this on one line, because a nice clean multiline thing isn't the kind of thing you do on the fly; it's what you do when you're cleaning it up to write about.) The '_sort -b_' bit is due to [[a GNU sort gotcha GNUSortGotcha]]. (Yes, [[I really write this sort of complex thing on the fly ../sysadmin/ACommandLine]].)