Chris's Wiki :: blog/sysadmin/LittleScriptsII

Another little script: `field`

April 29, 2006

In the footsteps of the first little script, here's something I call field:

#!/bin/sh
fn=""
for i in "$@"; do
  nf='$'$i
  if [ -z "$fn" ]; then
    fn="$nf"
  else
    fn="$fn, $nf"
  fi
done
exec awk "{print $fn}"

(The exec is another way around the Linux bash issue.)

You give it one or more field numbers, whereupon it reads standard input and prints just those fields to standard output. I wrote it because I got tired of typing 'awk "{print $7}"' and the like all the time.

Given field and howmany, we can now write what I'll call countup:

#!/bin/sh
field "$@" | howmany

Typical usage is things like 'countup 1 </var/log/web-xfers | sed 10q' to show me the top 10 IPs for today's web requests. (Today the big source is our internal Google search appliance. Although the third most active source was 200.55.156.97, poking us for security holes. Badly, which is the most annoying thing about it.)

(An ongoing index for all of my little scripts is here.)

Sidebar: an irritation

The irritating bit about field is all of the work it has to go through to generate the print instructions for awk. It feels like there should be a nice short Perl equivalent, but the closest I've come is the not quite correct:

#!/usr/bin/perl -w
use strict;
my @l;
while (<STDIN>) {
  chomp; @l = split;
  print join(" ", map {$l[$_]} @ARGV), "\n";
}

If you give this a field that doesn't exist in (some or all) records, you get complaints about 'use of uninitialized variable in ...'.

(There are more complicated constructs that will stop this, but I am interested in something compact, no larger than the Bourne shell script, and my Perl is sufficiently rusty that I can't see one right now.)

Comments on this page:

By DanielMartin at 2006-04-29 08:35:10:

See, around here we just use cut with appropriate -f and -d arguments. However, those awk scripts are nice for re-arranging the field order, which cut won't do.

As for your perl script:
#!/usr/bin/perl -wln
use strict;
use vars '@a';
BEGIN {
  while(@ARGV and @ARGV[0] =~ /^\d+$/) {
    push @a, shift;
  } 
}
sub fix{defined($_[0])?$_[0]:""}
my @l = split;
print join(" ", map {fix($l[$_ - 1])} @a);
(I can't just use map{$l[$_]||""} because that would drop fields that were a single "0")

This starts numbering fields at 1, just like your awk script (the other perl script started at 0) - if you don't like that it's a simple matter of wiping out a - 1. It also takes filenames as parameters in the grand unix tradition - any parameters that are just numbers are interpreted as fieldnames, and at the first parameter that isn't a number it starts reading files. (if all parameters are numbers, it reads from stdin)

Note that the -ln option combination is often what you want when you want perl to behave a bit like awk. (Actually, to get perl to behave more like awk you usually set $,=" " as well, which I could have done here, but I didn't think it added anything)

A little perl script I've had kicking around on a few machines here is this ditty:
> ruler 53
123456789_123456789_123456789_123456789_123456789_123
It just prints out a ruler that many characters long. I do a lot of work with files that are formatted into lines with fixed-width fields. It's useful to say something like:
> ruler 120; sed -n '55-60 p' VendorDataFile.dat

By cks at 2006-04-29 15:49:24:

Whoops, my bad for not catching the off by one issue with my Perl script; I did indeed want it to have the awk-style count from 1 behavior. (I don't know how I managed to miss that in my brief testing.)

My big problem with cut is that I actually want awk's default field separation behavior, which is just about impossible to get with cut. Cut is really great for /etc/passwd, but really bad for things with variable whitespace and the like.

(And indeed the argument handling in your script is exactly what I had in mind for a grown up version of it.)

Written on 29 April 2006.

«	Some first impressions of Fedora Core 5

Weekly spam summary on April 29th, 2006

Another little script: field

Sidebar: an irritation

Comments on this page:

Another little script: `field`