Wandering Thoughts archives

2011-09-25

How we handle iSCSI device names in Solaris

One of the quite important things we've done in our SAN environment is that our local software hides the real Solaris device names in favour of local labels for iSCSI devices. Hiding the real device names is not something that's normally recommended, but I feel that iSCSI is a special case, especially in a Solaris environment with MPxIO enabled.

First off, let me give you a concrete example that may illustrate why we hide device names. Here is the normal Solaris device name for a disk and the iSCSI name we use:

c2t49455400000000006F7377616C6430316935000000000000d0

oswald:disk01/0

(I think you can tell which is which.)

Yes, that's a huge device name. This is the kind of device names that you get with MPxIO, because MPxIO basically encodes a disk's identifying information in the official device name (as you might guess from the embedded hex). This is all well and good, but it means that if you use MPxIO you can easily have a whole bunch of very long names that differ in only small and basically opaque ways. Even with cut and paste it would be terribly easy to make a mistake, and I don't want to think about copying these names by hand.

The bigger problem is that Solaris device names for iSCSI devices have very low information content; with or without MPxIO, they're essentially opaque magic identifiers. To do anything much with them you need to reverse them back to determine what iSCSI target device they correspond to, because that's the only way to understand what you're doing or what's going on (or gone wrong). By contrast, our transformation of the names has very high information content; it immediately identifies the iSCSI backend, the specific physical disk on the backend, and the logical chunk from the disk.

In short, our hiding the device names transforms a mere identifier for the disk to the identity of the disk.

(This is not (and cannot be) a generic, site independent transformation. Our naming and remapping process only makes sense in our environment; in a different environment with a different iSCSI topology you would want to emphasize different things about the identity of the disk.)

There are drawbacks to doing this hiding; for a start, we need to do some extra work in scripts pretty much every time we deal with devices. Since Solaris doesn't directly export MPxIO mapping information for iSCSI devices in a machine readable format, this means we get to run iscsiadm and parse its output (which has the usual hazards). Still, I think that it's worth it; the visibility it gives us for what we're doing and what's going on is very important in practice.

(Most of our locally written tools degrade gracefully if they can't get this mapping information, displaying and accepting 'raw' Solaris device names. And of course we can always work directly with the ZFS commands, which of course only deal with Solaris device names.)

solaris/OurISCSINames written at 23:44:44; Add Comment

Danger signs for mail senders in SMTP conversations

This is another one of those entries that I write for people who are never going to read it, but I don't care; I just feel like pointing out the relatively obvious.

Suppose that you are someone who runs a mailing list service. Like everyone else who offers such a service, spammers will attempt to (ab)use it. Thus, one of the important things that you need to do is detect signs that you have a spammer's mailing list, and these days you certainly can't count on abuse complaints to tell you this.

As I've mentioned before, SMTP time rejections can be an important signal. The corollary of this is that the kind of SMTP rejection matters, and in particular you should really pay attention to MAIL FROM and DATA rejections and consider them a significant warning sign. This is because there are many fewer reasons for rejecting at those stages than for rejecting at RCPT TO time so if your mail is rejected then, well, there's any number of explanations besides 'it's spam'; the user's account could have expired, for example.

(And, let us admit, a disturbingly large number of mail systems have temporary glitches that cause equally temporary RCPT TO failures. This is why real mailing list management software pretty much never automatically removes addresses on a single RCPT TO failure.)

Since they don't have these relatively innocent explanations, mail rejections at MAIL FROM or especially from DATA are often signs of something serious going on. In particular a permanent failure at DATA time almost invariably means that the recipient's system really dislikes the message for some reason; if you're running a mailing list service, the usual case is that it's spam. A MAIL FROM rejection can have more innocent explanations, including a misconfigured MTA on the other side, but it is still more of a danger sign than a RCPT TO rejection.

(A significant volume of RCPT TO failures is still a danger sign, in part because it means that either the list of addresses is old or that the mailing list was badly maintained before it moved to your service. And if a mailing list has a few good mail-outs and then suddenly its RCPT TO failures spike upwards significantly, well, that's a bad sign itself. It could be that a whole bunch of user accounts just coincidentally got expired or filled up, but it's more likely that a bunch of anti-spam systems that reject at RCPT TO time suddenly woke up.)

Of course, all of this presumes that you are trying hard to run a 'clean' mailing list service instead of any of the various alternatives. I'm not convinced that there is or can be any such thing these days, as convenient as it would be for modern web applications if there was.

spam/SMTPDangerSigns written at 01:31:13; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.