Wandering Thoughts archives

Good editors aren't better or worse, just different

It's common to compare text editors and say that one is better or worse than another. But after a certain point this is wrong. Editors are often not better or worse but different than each other, each better at specific things. Such differences mean that there is no absolute scale of better and worse, and indeed that the question of which is a better editor in general is meaningless.

Let me illustrate this using the differences between the three editors that I use most frequently:

• vi is the best sysadmin editor, but it has distinct limits. For example, vi is somewhere between so-so and terrible at editing multiple files at once.

  (I inexplicably skipped this vi limitation before, perhaps because it is so much a part of vi that I never think about it, much like a fish in water.)

• Rob Pike's sam is the best graphical editor and the best editor for certain sorts of mass structured changes (within certain limits it has the best search and replace going, especially across multiple files). But it basically requires graphics and has nothing like smart indenting or the like.

• GNU Emacs is the best code editor, due to things like its freakishly superintelligent smart indenting and other extensions, but it is not suitable as a sysadmin editor and it is not as good a graphical editor as sam.

Looking at this it is hopefully clear how absurd it would be for me to claim that any of these editors was better or worse than any other one. They are all good editors, and they are all different tools that fit different situations best.

Note that a lot of this involves personal taste issues. For example, how sam works is very much to my tastes, but probably inspires violent loathing in some people. Good editors have (for lack of a better description) an aesthetic, and either you like that aesthetic or you don't (although, like music, you may find that an aesthetic grows on you over time; vi's aesthetic certainly has grown on me).

Sidebar: on IDEs and my editor list

If I did anything with Java or other highly structured languages, I expect that I would add Eclipse or some similarly smart IDE to this list; as it is I work in Python and C, neither of which are very amenable to such things. (And in the brief time I tried Eclipse, it had a distressing tendency to want to run a 'project', not just edit files for me. As a sysadmin, I can't make an IDE 'project' out of every package that I need to modify a bit.)

One consequence of mathematical security thinking

This realization struck me after writing yesterday's entry:

One consequence of the security paranoia way of thinking (or if you want to use the polite term, of the mathematically correct security way of thinking) is that false negatives are considered unacceptable. Either they are flaws in the implementation or they are flaws in the theory underlying the security system, and in either case they must be eliminated; it is incorrect to allow one to remain.

(By false negatives here I mean situations where the security system fails to detect something that is not secure, within the bounds of the things that the system is supposed to detect and deal with.)

The problem with this is that in practice, an unwillingness to accept false negatives almost always drives up the false positive rate, with predictable bad consequences in worlds inhabited by ordinary people (as opposed to Alice and Bob).

(In theory you can embed mathematical security perfection inside heuristic systems that sometimes let questionable things past in order to reduce the false positive rate. In practice, should you do this the mathematical security people generally pitch a fit about the 'security holes' in your system. This reminds me of attitudes on refactoring dynamic languages.)

Now, to be fair I'll give you a devil's advocate view of this: the problem with allowing small holes in computer security is automation. Automation means that any vulnerability, no matter how small, that can be automatically exploited winds up creating a huge amount of leverage against a system; a tiny chance or a tiny amount of return multiplied by enough actions equals an explosion. This implies that computer security must be much more airtight than physical security, where attacks are bounded by relatively low physical limits, and therefor we must insist on no false negatives, no matter the costs.

(Spot the problem with this argument for a no-prize.)

The problem with security alerts, and indeed all alerts

We know that in practice, users have been conditioned to treat security alerts and other caution dialogs as obstacles; they will blindly click whatever option makes the dialog get out of the way of whatever they're doing, no matter what. It has recently struck me that there is an obvious reason why, and it's even something I've written about before: it's the problem of false positives.

Unless you have really serious problems, your system is almost always doing what you want and you almost never are experiencing genuine security issues; in other words, true problems are very, very rare. Thus, a false positive rate that is at all detectable implies that almost all alerts are false positives. In an environment where even nine in ten alerts are false positives, it shouldn't be any surprise that users just click on things to make them go away. And nine in ten is being optimistic; I expect that the figure is more like 9,999 in ten thousand.

(I'm counting all 'caution! you might be doing something bad!' alerts in this figure, not just security alerts, partly because they all look and behave mostly the same to users.)

Unfortunately, fixing this is likely to be remarkably hard. Dealing with the false positive rate runs into the problem that further reducing an already small number is usually highly non-trivial, especially if you need to make it several orders of magnitude lower. That leaves reducing the number of 'events' in general; since you can't exactly tell users to do less with their computers, this is probably going to require fundamental rethinks that make entire classes of problems just impossible (so that they do not have to be checked and thus cannot have false positives).

(Undoable operations are one way of making entire classes of alerts disappear. Unfortunately it is hard to have undoable security sensitive operations; once your password is sent somewhere, well, it's sent.)

Why I hate 'security questions'

You know 'security questions'; they are the extra questions that websites (and other services) attach to your account that will theoretically let you regain access to the account if you forget your password. I really don't like them, and it's not because they are a terrible security idea. Well, not directly.

Hopefully everyone already knows the primary problem with security questions: if you pick questions and answers that you can remember, it's quite likely that an attacker can work out the answers themselves from public or semi-public information (especially if you are paranoid enough to be concerned about people that know you to at least some extent, not just random strangers). Even without that, it may be easier to guess answers to the security questions than to guess your password.

What this means is that security questions are effectively additional passwords to your account, and your answers need to follow all of the rules for passwords. So what security questions are is hard to remember, rarely used passwords. That I have to write down and keep track of, securely, and worry about.

Gee, thanks.

But that's just the obvious reason to hate security questions. The more subtle reason is the underlying reason why they exist in the first place. We can see this real reason by asking the question of what would happen if they didn't exist and you forgot your password: you'd go talk to customer service to sort out the situation. So security questions really exist so that companies don't have to have customer service people; in other words, they're making my life more painful (and reducing my security) in order to save themselves some money.

(The case of free services is one of those hard ones, since it might not be possible to provide the free services at all if you had to provide full customer support too. In some cases, such as free webmail providers, I would consider this a benefit.)

BitTorrent's file fragmentation problem

When BitTorrent receives a file, it gets the various chunks out of order, generally in a completely random one. This presents the client with the problem of putting them in order and in place.

I believe that historically there have been three approaches to deal with this. The very first BitTorrent clients did it the simple way: they put every received block in its correct place by seeking to that spot and writing the block out. The problem with this was horrible file fragmentation (resulting in terrible sequential read performance); because the blocks were written in random order they were generally allocated randomly around the disk, instead of nicely sequential.

Next came the approach of always growing the file in order, and reordering blocks inside the file. When the client got block N, it initially wrote it at the current end of file; when the file grew to be more than N blocks long, block N finally could be swapped into its correct location in exchange for whatever was already there. This avoids file fragmentation (the client is always expanding the file sequentially), but at at the cost of an increasing amount of file IO to shuffle blocks into their correct places.

(This file IO does not matter all that much for typical clients, which have much more disk bandwidth than network bandwidth, but it can be a significant issue if you are running BitTorrent on fast networks, especially since disks are seek limited.)

The final approach is for the client to pre-write the file (with empty contents) before it starts receiving anything, and then to directly write received blocks into their correct locations. Pre-writing the file forces sequential allocation (possibly better than growing the file does), and rewriting parts of it later generally doesn't change this. The cost of this approach is a potentially significant startup delay, as the client writes what may be several gigabytes to disk.

(Note that many of these sequential allocation assumptions break down if you are using a log-structured filesystem such as ZFS. Copying the file again after you've received it may be the only good solution.)

I wish I could tell you that BitTorrent has solved this problem, but as far as I know it hasn't; you just get to pick which drawback you want. I believe that most BitTorrent clients today default to the second approach but give you an option to do the third.