Wandering Thoughts archives

Why XML is terrible for configuration files

There's a lot of things that get called 'configuration files', so I want to be specific that I mean the sort of configuration files that have three primary uses: they're written by people, used by programs, and later read by people who are trying to figure out what the programs are set up to do. These are the kind of configuration files that XML is terrible for.

(There's a whole ecology of 'configuration files' that are generated by one program and consumed by others, and are pretty much never touched by people. I don't care what format they're in, and there's perfectly sensible reasons to use XML for them.)

The reasons that XML is terrible for configuration files are right there in the description of what these sorts of configuration files are for. The only one of those three things that XML makes easy is being used by programs; XML is famously difficult and annoying to write by hand and equally hard to read (partly because it is so verbose; excess verbosity causes people to lose track of where they are). Common ways to structure data in XML files make this even worse because they tend to be designed for the convenience of programs, not to be comprehensible to people.

General XML editors improve the situation somewhat but I feel that they don't really do all that much for making it genuinely easy for people to write and read XML. This goes doubly so if your XML format has data structuring issues.

XML is also prone to a particular disease, which is best illustrated by asking a question: is your XML format actually documented, in full detail, in a way that is at least as good as the Atom feed format specification or the description of your favorite program's regular configuration file? All too often the answer is that it is not, because people have the peculiar impression that using XML with verbose element and attribute names plus some sketchy documentation is sufficient.

(Please note that a DTD is not documentation. Try again.)

This issue cannot be solved by creating a nice user-friendly program to create and maintain the XML configuration file. If you do this, what you have really done is created a program without a real configuration file that is instead configured only through an GUI interface. And you still have the documentation problem; it's just that you now have to document the effects of the program instead of the configuration file.

(For bonus points, this configuration process is generally asynchronous so you can't immediately see the effects of your configuration changes.)

The dividing line between supporting code and forking it

Suppose that you want to provide support for a bunch of open source code but are not the primary author or maintainer. Inevitably you will have to patch bugs and add features on your own in some way (and perhaps fix the code to port it to your environment or whatever). This leads to a question: where is the dividing line between merely supporting the code and actually forking your own version of the code?

To me, the dividing line is whether you can get your changes accepted upstream and applied to the main codebase (assuming that you even try). If you can't get changes accepted upstream, you have to maintain your own version of the code, forward-porting all of your fixes and changes to new versions of the main code that you chose to adopt (or perhaps porting changes from the main code into your version). Ergo, forking.

If you can get changes accepted upstream you have less divergence, especially over the long term. Instead of carrying changes that have to live forever and be perpetually forward-ported, you are instead simply developing and maintaining changes until they can be merged, which is part of normal development for many projects. (Of course, ideally you merge changes upstream as fast as possible; the faster you merge, the less work each of your changes is for you.)

The direct corollary is that your ability to merely support code instead of forking it depends on how willing the upstream is to accept your changes. A closed or mostly closed upstream forces you to effectively fork, whether you like it or not. And of course this presumes that you can find an upstream and that the upstream is alive.

(In real life many people doing 'support' for open source projects carry a mix of changes that are in the process of going upstream and changes that the upstream will never accept for various reasons. Note that a lot of people do this sort of support in the open source world, as this issue applies to lots of people who packages programs for particular operating systems.)

The many IPv6 addresses of an IPv4 machine

Ignoring things like link-local addresses, a dual-stack machine with an IPv4 address can have and see at least four different sorts of IPv6 addresses:

1. what I will call non-legacy IPv6 addresses. These are only reachable through the 6bone.

   (I think these are sometimes called 'native IPv6 addresses'.)

2. 6to4 addresses in 2002::/16. These are reachable both through the 6bone and also directly over IPv4 (using 6in4 to encapsulate IPv6) by other dual-stack machines that have brought up routing for 2002::/16 through an open-ended SIT connection.

3. 'IPv4 compatible addresses', ::<IPv4 address>. These are not reachable via the 6bone, but are reachable by any dual-stack machine that has brought up an open-ended SIT connection or tunnel, even without any particular IPv6 routing. On Linux you can do this with nothing more than:

   ip tunnel add 6to4 mode sit remote any local <your-IP>

   I believe that any machine using 6to4 will also be reachable this way.

   Since these still use IPv6, you cannot use this form of address to talk to arbitrary IPv4 machines.

4. 'IPv4 mapped addresses', ::ffff:<IPv4 address>. Although they are using IPv6 APIs, programs seeing or using these IPv6 addresses are actually talking IPv4. Not all systems and not all programs will necessarily support IPv4 mapped addresses; some may insist that using IPv6 APIs results in actual IPv6 conversations.

   Because these are real IPv6 addresses being handled by real IPv6 APIs, programs using them can fail when they're running on machines where IPv6 is disabled; for example, 'ssh ::ffff:<IP>' can fail on a machine where 'ssh <IP>' works. This can make it awkward to use these IP addresses pervasively.

#1 and #2 are the only addresses that are reachable by pure IPv6 machines. #1, #2, and #3 use IPv6 only (although traffic may be transported over the IPv4 Internet); #4 uses IPv4 only. The only IPv6 addresses that can be automatically determined for a remote machine given only its IPv4 address are #3 and #4; while you know the /48 prefix that a 6to4 machine will use, you don't know the host portion of the address in advance.

(In many cases a host portion of ::1 is a good bet, but you don't know for sure.)

A machine can have all three of the 'real' sorts of IPv6 addresses active at the same time, and there are vaguely rational reasons to do so. If you run your IPv6 default route through your non-legacy IPv6 connection your only use of 6to4 will be to talk directly to other 6to4 machines, which may be a good thing.

(This is definitely the sort of entry I write to get this all straight in my head.)

Some thoughts about 6to4

I've previously written about the problems with the IPv4 to IPv6 transition, and how the IPv6 people could have made it easier. Well, it turns out that I have to take back some of the grumpy things that I said back then because I just found out about 6to4, which makes IPv6 connectivity fairly trivial for machines with public IPv4 addresses (and generally it works), partly by embedding the IPv4 address space into IPv6.

(I'll let you read about the details in the Wikipedia entry, which describes it better than I could. See here for one description of how to set it up on Linux.)

6to4 is one of those things that strike me as a neat hack, in the classic sense; it is an elegant, clever trick to get something done. The simplicity of how things work is very nice and it avoids all sorts of annoyances with more complex VPN/tunnel systems. Unfortunately, I think that in the long term 6to4 is probably doomed; it's useful for easy experimentation, but not really as a way to get large numbers of people onto the IPv6 Internet. The problem is the issue of gateways.

(The remainder of this assumes that you understand how 6to4 works.)

Two 6to4 machines will talk directly to each other over the IPv4 Internet, but to talk to general IPv6 addresses they have to route through a 6to4-to-IPv6 gateway. Because the IPv6 people had no desire to import the IPv4 routing table into IPv6 routing tables, 6to4 gateways can only advertise the entire (IPv6 embedded) IPv4 address space, not subset routes for parts of it.

An ISP with top-level IPv6 connectivity can easily arrange for outgoing 6to4 traffic from its customers to go through its own 6to4 to IPv6 gateway; it just routes traffic for the magic 192.88.99.1 gateway IP address to its gateway. However, it has no way of causing the return traffic from pure IPv6 nodes to flow through its gateway; instead, the return packets will be routed through whichever 6to4 gateway is closest (in a network sense) to the IPv6 machine at the other end.

Any significant use of 6to4 thus necessarily starts shunting more and more traffic through public gateways, even under the best of circumstances. Many of these gateways will not be located on what are the natural network paths between the end nodes, where the traffic would have flowed regardless of where the gatewaying happened; instead, the traffic will be diverted out of line, resulting in extra volume transiting into and out of unrelated networks. This effect is magnified by ISPs who do not operate 6to4 gateways (whether public or private), because then even the outgoing 6to4 traffic is running off through another network.

For end users, this results in packet delays. For the ISPs involved, this causes issues with both link volumes and peering politics; you're spending money to carry other people's traffic out of the goodness of your heart, and your peers may not like that anyways. Something is likely to give sooner or later, and so the more popular 6to4 gets, the fewer truly public gateways I suspect we'll see and the higher the chance that you won't get full and reliable IPv6 connectivity with 6to4.

This doesn't make 6to4 useless; it's still a great way to start playing around with IPv6 without having to find a 6bone tunnel broker, registering, setting things up, and all of that. I just can't see it as a long-term compatibility solution for general interaction between IPv4 and IPv6 machines, unless there's something I'm missing.

(I looked into IPv6 tunnel brokers once upon a time and concluded that it was too much work. The great appeal of 6to4 is that all I have to do is run a few commands locally and bang, I have IPv6 connectivity and can start finding out all of the programs that now break.)

Note that even today this means that your 6to4-enabled machine may not have full IPv6 connectivity; what connectivity you get depends on the administrative policies and routing of the gateways involved. Possibly this means I should grit my teeth and get a connection through an IPv6 tunnel broker.

(I expect that these issues are already well known in the 6to4 and IPv6 community; I'm just new to all of this IPv6 stuff.)

Sidebar: why the 'no subset routes' thing hurts

If 6to4 gateways could advertise IPv6 routes for a subset of the IPv4 address space, your local ISP's 6to4 gateway could announce itself as the IPv6 gateway for only its IPv4 address space. This would cause traffic to flow the way it should; the returning IPv6 packets would transit over the 6bone until they reached your ISP, instead of flying off to some other 6to4 gateway to be turned into IPv4 packets.

This would have the additional effect of making the packet routes symmetrical. Right now, 6to4 connections are assymmetrically routed, where the outgoing packets take one path and incoming packets take another. In fact, incoming ones may take any number of paths depending on where they're coming from.

Why a laptop is not likely to be my primary machine any time soon

I know and read a number of people who use laptops as their primary machines, but I'm one of the people who's not interested in the idea (even ignoring any issues of relative prices). I wound up actually thinking about the question recently, and as it turns out I think I have a fairly odd set of reasons for it.

So, here they are so far:

• I have very particular tastes in keyboards (I have used a BTC-5100C keyboard for more than a decade) and for the space immediately in front of the keyboard. Laptops may have decent keyboards, but they don't have my keyboard.

• I want a fairly physically large display with good resolution, especially good vertical resolution; when there's room for it, I want two of them.

• I use two drives in my systems in order to have mirrored (system) disks. (Of course this can have drawbacks.)

In the past, my desire for Unix (ideally Linux) would also have been a significant obstacle, but my impression is that it's now relatively easy to find a nice modern laptop that has good Linux support. (Hopefully I'm not wrong.)

Another way of thinking about this is that I have two roles for computers: the computer I sit in front of all the time, and the computer that I take places for relatively moderate use. For the heavily used computer, I have strong and very particular opinions about the pieces of the computer that I interact with a lot (the keyboard, the displays), but I'm indifferent to the rest of it (provided that it's quiet). I don't care as much about the casual computer, but I want it to be small, light, and still nice for productive work.

(The late Dell Mini 12 is about my platonic ideal of the casual laptop in form factor, screen resolution, and keyboard.)

It's pretty clear to me that some of these desires clash even in the best of circumstances, particularly the displays; a laptop screen big enough to be one of my regular displays makes the laptop too big to be conveniently portable. Thus, if I tried to use a laptop for both roles the only use I'd get for it in the full time usage role would be as the system unit of a desktop system, as I wouldn't use either its display or its keyboard (and I'd still only have one system disk). If I absolutely had to have only computer this could be workable, but if not, there's little advantage to it.

I suspect that other people are generally much less particular and picky about their keyboards, displays, software, and so on. (Or, alternately, they have found a laptop maker whose keyboards and screens they are as fond of as I am fond of my favorites.)

(This entry was sparked by the discussion here. Plus, I feel like not writing about documentation for days on end.)

What charging credit cards doesn't prove

Every so often, commonly in the context of SSL certificates, someone puts forward the theory that charging money for things makes the customers somehow more identifiable and reliable than giving it to people for free (with the same other authentication of customers). After all, so the theory goes, when you give people something just because they have a particular email address, that's not much, but when you've charged their credit card, you have a lot more confidence in their real identity.

This is wrong. To explain why it is wrong, let's talk specifically about SSL certificates.

The basic model of 'verifying' SSL certificates is that in order to get a certificate for a domain, you have to prove that you (theoretically) have power over that domain; you have one of a certain number of email addresses at that domain, you can put things on its web server, or something of the like. Most SSL certificate authorities also charge money on top of this; you submit credit card information along with your Certificate Signing Request, they charge your card, and if the charge goes through you get your signed certificate in email. By collecting money from you, they've gotten a stronger verification than before.

Except that they haven't, because I snuck a fast one into this description: charging a credit card is not the same as actually collecting money from it. No SSL CA waits on giving you your certificate until they actually have received your money from the credit card company; the delays involved in that would drive most customers away. Instead they issue SSL certificates very close to on the spot, which means that SSL CAs are not verifying that you can pay them money, they are verifying that they can charge a credit card. And there are a lot of ways to get a credit card number that can have some amount of money charged to it and not have that reversed, rejected, or detected as fraudulent for (say) six hours, if not days.

(Oh, sure, once the charge blows up the SSL CA will try to revoke the SSL certificate. Good luck with that.)

(This is kind of a reaction to this, because I think this misapprehension is a general one.)