Wandering Thoughts archives

2018-05-15

I have a boring desktop and I think I'm okay with that

Every so often I wind up either reading about or looking at pictures of beautifully customized Unix desktops. These are from people who have carefully picked colors and themes, often set up a highly customized window manager environment, set up all sorts of panels and information widgets, and so on (one endless source of these is on reddit). Sometimes this involves using a tiling window manager with various extension programs. I look at these things partly because there's something in me that feels drawn to them and that envies those setups.

My desktop is unconventional, but within that it's boring. It has colours that are either mismatched or at best vaguely matched, various font choices picked somewhat at random, and assorted decorations and so on that are there mostly because they're what's easy to do in the plain old standby of fvwm. There's very little design and very little use of interesting programs; I mean, I still use xclock and xload, and I don't think fvwm is considered an interesting window manager these days.

(Fvwm certainly has limitations in terms of what you can easily do in it. A dedicated person could expand fvwm's capabilities by use of modern X programs like wmutils and wmctrl, but I'm not such a person.)

I wound up thinking about this when I was recently reading another article about this (via, itself via), and this time around I came to a straightforward realization, one that I could have arrived at years ago: I'm not that dedicated and I don't care that much. My mismatched, assorted desktop is boring, but it works okay for me, and I've become the kind of pragmatic person that is okay with that.

I wouldn't mind a nicer desktop and every so often I make a little stab in that direction (I recently added some fvwm key bindings that were inspired by Cinnamon), but I'm never going to do the kind of work that's required to build a coherent custom desktop or make the kind of sacrifices required. Tiling window managers, programmable window managers, highly custom status bars, all of that stuff is neat to read about and look at, but it's not something for me. The best I'm likely to ever do is minor changes around the edges (at least until Wayland forces me to start over from scratch). And so my desktop is always going to be boring. I think I'm finally okay with that.

(There's a freedom in giving up in this sense. One way to put it is that I can stop feeling slightly guilty about not having a nicer, more coherent desktop environment, or in having something that's the kind of advanced environment you might expect a serious Unix person to have. I know this is an irrational feeling, but no one said feelings are rational.)

PS: This also means that I can give up thinking about switching to another window manager. It's quite unlikely I could find one that I want to use other than fvwm (fvwm's icon manager is extremely important to me), but I've long had the thought that there might be a better one out there somewhere. Maybe there is, but even if there is, it's probably way too much effort to switch.

MyBoringDesktop written at 01:20:46; Add Comment

2018-05-03

Using grep to hunt around for null bytes in text files

Suppose, not entirely hypothetically, that you've developed a suspicion that some mailbox files have zero (null) bytes in them. Null bytes are not traditionally found in mail messages, or indeed in any text format, and their presence is not infrequently a sign that something has gone wrong (in email related things, an obvious suspect is locking issues). So if you suspect null bytes might be lurking you want to at least check for them and perhaps count them, and once you've found some null bytes you may want to know more about the context they occur in.

On most modern Unixes, searching for null bytes is most easily done with GNU grep's 'grep -P' option, which lets you supply a Perl-style regular expression that can include a direct byte value:

grep -l -P '\x00' ....

If your version of grep doesn't support -P (FreeBSD's doesn't), you'll have to investigate more elaborate approaches. The general problem is that a lot of things will interpret a literal null byte as a C end-of-string; you need to find a way to supply one to your grep that doesn't fall victim to this, and then hope your (e)grep does the right thing.

(Really, it might be easier to get and compile the latest GNU grep just for this.)

Once you've found files with null bytes, you might want to do things like count how many null bytes you have and in how many different places. As I found out recently, modern versions of awk are perfectly happy about null bytes in their input, which makes life reasonably easy when combined with 'grep -o':

grep -ao -P '\x00+' FILE |
 awk '{cnt += 1; tlen += length($0)}
      END {print cnt, tlen}'

(More elaborate analysis is up to you. I looked at shortest, longest, and average size.)

To really dig into what's going on, you need to see the context that these null bytes occur in. In an ideal world, less would let you search for null bytes, so you could just do 'less FILE', find the first null byte, and go look around as usual. Unfortunately less has no such feature as far as I know, and neither does any other pager. I will save you the effort and say that the easiest way to do this is to use grep, telling it to provide some context:

grep -a -n -C 1 -P '\x00' FILE | less

It's worth breaking down why we're doing all of this. First, we're feeding the grep's output to less because less will actually show us the null byte or bytes, instead of silently not printing it. Grep's -P argument we're already familiar with. -a forces grep to consider the file printable, despite the null bytes. -C N is how many lines of context we want before and after the line with null bytes. Finally, -n prints the line numbers involved. We want the line numbers because with them, we can do 'less FILE' and then jump to the spot with the null bytes using the line number.

When looking at the output from less here, remember that null bytes will be printed as two characters (as '^@') even though they're only a single character. Where this came up for me was that one of our null bytes was in some base64 lines, and initially I was going to say that the null byte was clearly an addition to it because the line with it in was a character longer than the other base64 lines around it. Then I realized this expansion, and thus that our null byte had replaced a base64 character instead of being inserted between two.

(Unfortunately all of this looking brought us not much closer to having some idea of why null bytes are showing up in people's mailboxes, although there's indications that some of them may have been inserted by the original sender and then passed intact through Exim. What I haven't done yet and should do is actually test how various elements of our overall mail system behave when fed SMTP messages with null bytes, although that would only tell me the system's current behavior, not what it used to do several or many years ago, which is when some of the nulls apparently date from.)

PS: I've tried loading the mailbox file into an editor to use it to search for nulls. For a sufficiently large mailbox, the result didn't go all that well, and I had to worry about inadvertently modifying the file. Perhaps there is an 'editor' that is efficient for this, but if so I don't have it lying around, while grep is right there.

(I believe I got the grep -P '\x00' trick from Stackoverflow, where it's shown up in a number of answers.)

GreppingForNullBytes written at 00:38:58; Add Comment

By day for May 2018: 3 15; before May; after May.

Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.