Why indirect xdm probably doesn't work on your Linux machine

We recently had a fun adventure of debugging indirect XDMCP on our Ubuntu LTS 6.06 (Dapper Drake) servers. The problem was that you couldn't go from the chooser on one server to logging in on another server, although you could log in to the machine the chooser was coming from.

It turns out that this is broken by default in pretty much every modern Linux distribution, and the culprit is IPv6 support. In fact it makes a good illustration of why adding IPv6 support is not an easy slamdunk. The bug goes like this:

• when you pick a machine in the chooser, the chooser server sends the target machine a UDP packet saying basically 'I am forwarding you a request to manage display Y on IP address X'.

• xdm prefers to bind to the IPv6 version of the XDMCP UDP port, when it can.

• the code that receives and processes forwarded requests insists that the address of the machine it is being asked to manage is the same length as addresses of the address family that the listening socket is for.

• IP addresses are not the same length as IPv6 addresses, so when xdm receives a request to manage an IP address over its IPv6 UDP socket, it discards it as an error. (Cleverly, it discards it without so much as a debugging message to complain about the problem.)

Reading the source code, it is easy to see how the bug was introduced. Originally the code dealt with only IP(v4) and Unix domain sockets; in this context it was sensible for the IP socket to insist that it was being asked to manage a 4-byte address, not something that looked more like a Unix domain socket address. When IPv6 support was introduced, it was almost certainly done by just copy and pasting the IPv4 code and changing the IPv4 specific things to IPv6 things.

(The code itself is cut & paste identical, but I can't be sure how it came about.)

This illustrates one of the problems of the would-be IPv6 migration: you can't just mechanically add IPv6 support to programs, you have to look through them to pick up more subtle implications of the whole change. In the old days of xdm, it made no sense to get a Unix domain socket address through an IP forwarded request or vice versa; in the new era, it does make sense to get an IPv4 address through an IPv6 forwarded request (and it may make sense the other way around, too; you'd have to think about it). This is not just adding support mechanically, this is a logic change.

(The bug is still there in xdm 1.1.4 and the current development version available through gitweb.freedesktop.org . Interested parties can look at the routine forward_respond in xdmcp.c.)

Sidebar: how to work around this

There are two ways to work around this:

• make xdm listen for XDMCP packets only on the IPv4 INADRR_ANY, not the IPv6 one. You do this by adding the line

  LISTEN 0.0.0.0

  to /etc/X11/xdm/Xaccess, which controls various bits of XDMCP.

• turn off IPv6 entirely. You can only do this if IPv6 was built as a module in your kernel, but it probably was. You want to blacklist the module net-pf-10; you can probably do this by adding a file to /etc/modprobe.d that contains:

  alias net-pf-10 off

Unless you're one of a rare band of people, you aren't losing anything by turning IPv6 off entirely except some future aggravation with the next program that has subtly buggy IPv6 support.