Wandering Thoughts archives


The evolution of our account creation script

One of the things about system administration automation is that its evolution often follows the path of least resistance. This can leave you with interesting and peculiar historical remnants, and it can also create situations where it takes a relatively long time before a system does the obvious thing. As it happens, I have a story about this.

To go with our account request system, which handles people requesting new accounts and authorizing requested accounts, we have an actual script that we run to actually create Unix accounts. Until relatively recently that script asked you a bunch of questions, although they all had default answers that we'd accept essentially all of the time. The presence of these questions was both a historical remnant of the path that the script took and an illustration of how unquestioningly acclimatized we can all become to what we think of as 'normal'.

We have been running Unix systems and creating accounts on them for a very long time, and in particular we've been doing this since before the World Wide Web existed and was readily accessible. Back in the beginning of things, accounts were requested on printed forms; graduate students and suchlike filled out the form with the information, got their account sponsors to sign it, handed it to the system staff, and the system staff typed all of the information into a script that asked us questions like 'login?', 'name?', 'Unix group?', 'research group affiliation?', and so on.

At a certain point, the web became enough of a thing that having a CGI version of our paper account request form was an obvious thing to do. Not everyone was going to use the CGI form (or be able to), and anyway we already had the account creation script that knew all of the magic required to properly create an account around here, so we adopted the existing script to also work with the CGI. The CGI wrote out the submitted information into a file (basically as setting shell environment variables) and this file was then loaded into the account creation script as the default answers to many of the questions that had originally been fields on the printed form. If the submitted information was good, you could just hit Return through many of the questions. After you created the account, you then had to email some important information about it (especially the temporary password) off to the person it was for; you did this by hand, because you generated the random password by hand outside of the script.

(For reasons lost to history, the data file that the CGI wrote and the script loaded was a m4 file that was then processed through m4 to create shell variable assignments.)

When we wrote our account request system to replace the basic CGI (and the workflow around it, which involved manually emailing account sponsors to ask them about approving accounts), the simple and easy way for it to actually get accounts created was to carefully write the same data file that the CGI had used (m4isms and all). The account request script remained basically unchanged, and in particular it kept asking us to confirm all of the 'default' answers, ie all of the information that the account request system had already validated and generated. More than that, we added a few more bits of special handling for some accounts, with their own questions.

(Although the account request system was created in 2011, it took until a 2016 major revision for a new version of Django for us to switch from generating m4 data files to just directly generating shell variable assignments that the script directly sourced with the . command.)

That we had to actually answer these questions and then write the 'you have a new account' email made the whole process of creating an account a tedious thing. You couldn't just start the script and go away for a while; you had to periodically interact with it, hitting Return, generating a password in another window and pasting it in to the password prompt, and composing email yourself. None of these things were actually necessary for the backend of the account request system, but they stayed for historical reasons (and because we needed them occasionally, because some accounts are created outside of the account request system). And we, the people who used the script, were so acclimatized to this situation that we didn't really think about it; in fact I built my own automation around writing the 'you have a new account' form email.

At this point I've forgotten what the exact trigger event was, but last year around this time, in the middle of creating a bunch of new graduate student accounts (where the existing script's behavior was at its most tedious), we realized that this could be fixed. I'll quote my commit messages:

New 'fast create' mode for account creation that takes all the defaults and doesn't bother asking if we're really sure.

For fast mode, add the ability to randomly generate or set the initial password at the start of the process.

offer to send new-account greeting email.
make sending greeting email be the default (if you just hit return).

(In theory we could make sending the greeting email happen automatically. In practice, asking a final question gives us an opportunity to look back at all the messages printed out just in case there's some problem that the script didn't catch and we want to pause to fix things up.)

This simple change has been a real quality of life improvement for us, turning a tedious slog into a mostly fire and forget exercise that we can casually run through. That it took so long to make our account creation script behave this way is an illustration not just of the power of historical paths but also of the power of habituation. We were so used to how the existing system worked that we never really questioned if it had to be that way; we just grumbled and accepted it.

(This is, in a sense, part of the power of historical paths. The path that something took to get where it is shapes what we see as 'normal' and 'just how things are', because it's what we get used to.)

Sidebar: There were some additional steps in there

There are a few questions in the account creation script where in theory we have a genuine choice to make; for example, some accounts have several options for what filesystem they get created in. Part of what made the no-questions version of the script possible was that we realized that in practice we always made a particular choice (for filesystems, we always picked the one with the most free space), so we revised the script to make this choice the default answer.

Had we not worked out default answers for all of these questions, we couldn't have made the creation script not even ask the questions. We might have done both at the same time if it was necessary, but in practice it certainly helped that everything already had default answers so the 'fast create' mode could just be 'take all of the default answers without requiring confirmation'.

sysadmin/AccountCreationScriptEvolution written at 22:58:21; Add Comment

A recent spate of ZIP attachments with everything

Our program for logging email attachment type information looks inside .zip and .jar archives, including one level of nesting. Often what we see in this is routine, with basically the sort of content you'd expect from either routine stuff or malware, but recently we've been seeing zip archives that are just stuffed with at least one of almost any file extension you can think of. A few days ago we logged an extreme example:

1fnnAC-0003dZ-EP attachment application/zip; MIME file ext: .zip; zip exts: .jar; inner zip exts: .abc .abl .acc .ach .adc .adz[2] .afd .age .ago .agy .aht .ake .ala .alp .and .ans .aob[2] .aor .app .apt .ara .ary .aud .aus .ave .axe .baa .bag .bap .bat .bde .bet .bin .bis .bkg .boe .bra .bsh .buz .bye .cai .cal .cat .caw .cdg .chm .cit .class[10] .cli[2] .clo .col .cop .cpl .crc .crs .cst .ctg .cto .cup .cwt .dad .dbl .dcb .der .det[2] .dew .dey .dig .dil[3] .dks[2] .dur .dwt .dye .eft .ego .elb[2] .elm .els[2] .emf .emm[2] .emu .err .esd .esq .ext .eyn .fax .fbi[2] .fcs .fee .fei .fem .ffa .fgn .fig .flb .fly[3] .foe .fog .fud .gab .gae .gal .gas .geb .gig .gin .gio[2] .goa .gob .god .gon .goo .gox .gtc .gun .had[2] .hah .hak[2] .hao .hat .hau .hcb .hcl[2] .hed .heh .hen[2] .hes[3] .hia .hip .hir .hld .hoc .hoe .hts .hug .hye .ibo .ide .ihp[2] .ijo .ilk .imu .ing[2] .ipr[2] .iqs .ire .iwa .iyo[2] .jah .jap .jay .jct .jem[2] .jud .jur .kat .kaw .kay .key .khi .kop .kor .kos .kph .kyl .lab[3] .lap .lcm .lea .lek .les .lib .lid .lit .llb .lou .lub .lxx .mao .map .maw .meu .mf .mix .mks .mog .mor .mot .mph .mus .nee .nef .nei .nep .nut .oak[2] .obb .ofo .oki .one .oni .ops .ora .our .pan .pap .par .paw .pax .pay .pdq .peh .pep .pia .pie .pig .pit .pks .poh .pos .pot .ppa .pps .pre .pry[2] .psi .pwr .pyr .rab .ram .rat .raw .rct .ref .reg .res .rfs .rig .rim .rix .rld .roc .roi .rpm .rut .rux .rwd .rwy .rye .sab .sau .sds .sed[2] .sei .sel .sew .she .shr .sie .sil .sim .sip .six .sny .soe .sou .soy .sqq .stg .sum .sur[2] .syd .tar .tat .tay .ted .tef .tem .tng .ton .tou .twa .udo .uns .urb .urn .uti .vac[2] .vil .von .vum .wab .wae .wea .wop[2] .wot .wro[2] .wud .xii[2] .xiv .xxi .xxv .xxx .yam[2] .yay .yea .yeo .yer .yez .yoe .yrs .yun .zat .zen .zho .zig .zip .zod

(We deliberately log file extensions inside zip archives in alphabetical order, so it may well have had a much different order originally.)

This particular message was detected by Sophos PureMessage as 'Mal/DrodZp-A', which may be a relatively generic name. The Subject: of the message was the relatively generic 'Re: Invoice/Receipt', and I don't know what the overall MIME filename of the .zip was claimed to be. We've received a bunch of very similar attachments that were just .jars (not .zip in .jar) with giant lists of extensions. Many of them have been rejected for containing (nominal) bad file types, and their MIME filenames have been things like 'ORIGIAL SHIPPING DOCUMENTS.qrypted.jar' and "0042133704 _ PDF.jar".

(It's possible that these direct .jars would also be detected as Mal/DrodZp-A, but we reject for bad file types before we check for known viruses.)

I doubt that the attachment had genuine examples of these file types, especially things like .rpm (RPM packages) and .nef (Nikon camera RAWs, which are invariably anywhere from several megabytes to tens of megabytes for the latest high-resolution Nikon DSLRs). I'm sure that the malware has some reason for doing this spray of files and file extensions, but I have no idea what it might be. If there are some anti-virus products that give up if a .jar has enough different file extensions in it, that's kind of sad (among other things).

Sadly for any additional filtering we might considering doing, I suspect that the dangerous parts of this were in the actual Java stuff (eg the .class files) and everything else is distraction. It'd be somewhat interesting to pick through a captured sample, because I am curious about what's in all of those files (or if they're just zero-length ones put in to pad things out) and also what file names they have. Did the malware make up some jumble of random file names, or is it embedded a message in them or something clever? I'll never know, because it's not important enough to bother doing anything special for.

spam/ZipAttachmentWithEverything written at 00:26:29; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.