Wandering Thoughts archives

2020-06-26

NetworkManager and (not) dealing with conflicting network connections

I recently tweeted a wish for NetworkManager:

I wish there was some straightforward way to tell NetworkManager to not automatically connect to any wifi networks if my laptop has a wired network connection, while still auto-connecting if there is no wired Ethernet.

In NetworkManager you can set a priority for network connections, but as far as I can tell you can't tell it that two connections conflict with each other and should never be brought up at the same time. You can write scripts that run when connections change and that take down connections (on Twitter @zigford pointed me to a script for this here), but I don't consider this straightforward. So let me give you the story of how I have wound up wanting this.

I have a work laptop, which I've brought home periodically for short periods of light use in the past (for example over our Christmas vacations). At that point I set it up for wireless networking and to automatically connect to my home wireless for convenience. Then, thanks to current world and local events, I took my work laptop home for extended use over a longer period, and soon discovered that my home wireless network has surprisingly high and variable latency. Fortunately I'd also taken the laptop's USB Ethernet adapter and I have a little home switch, so I could change over to having my laptop use a wired connection (although not without fun discoveries).

As it happens, I don't have two home subnets, one wired and one wireless; I have one that everything is on (and the AP just extends it out from wired to wireless). When I initially set up my laptop at home on the wireless, I gave it a fixed IP address and name so that I could easily SSH to it (and because I give everything here a fixed IP address). When I 'switched' my laptop to using a wired connection, I gave that wired Ethernet address the same fixed IP address, because I had no desire to have to SSH to 'laptop-wired' versus 'laptop-wifi' (I just want to SSH to 'laptop'). Unfortunately this means that if both the wired and the wireless connections are active at once, both get the same IP and then fun things happen. Especially, some of my traffic to my laptop goes over the wireless, with increased and variable latencies, which is what I set up a wired connection to avoid.

(I'm honestly surprised that my DHCP server didn't object to handing out the same IP at once to two different things, but then I did tell it that both the wired and the wireless Ethernet addresses could have the same IP. I'm also surprised at how long it took me to notice this; I only did because I was running 'ifconfig -a' for another reason and noticed that my wifi adapter had an IP assigned.)

My current solution is to tell NetworkManager to not automatically connect to my home wireless network. This is less convenient if I want to use my work laptop from somewhere else, but in practice I almost never do (my work laptop is mostly used for video conferencing, since it has a camera and a microphone; actual work happens from my home desktop).

NetworkManagerConnectionConflict written at 22:42:14; Add Comment

2020-06-19

Removing unmaintained packages from your Fedora machine should require explicitly opting in

Ben Cotton recently wrote an entry on Fedora potentially removing unmaintained packages from your system under some circumstances, because there is a Fedora change proposing to remove 'retired' packages. The change proposal contains the following remarks:

Upgrade/compatibility impact

During an upgrade, all retired packages will be automatically removed.

[...]

How To Test

1. Upgrade to next version of Fedora. 2. Check all retired packages are removed.

In the ending of Ben Cotton's article, he says in passing "[B]ut we have to make sure we’re clearly communicating what is happening to the user" if package removal happens. I will go further than that.

Removing packages from your system on Fedora upgrades should require an explicit opt-in, and this opt-in should be able to show you the list of packages being removed.

Going beyond that, Fedora should never remove unmaintained packages from your system without this opt in, for example they should never push out an updated fedora-retired-packages RPM in Fedora updates.

Removing unmaintained packages from people's systems is removing functionality with no replacement or equivalent. This can break what people are doing with their Fedora machines, and doing so is both morally wrong and dangerous in practice. It doesn't take too many cases of Fedora upgrades or Fedora package updates breaking things without warning for people to stop doing either of them.

Because this requires explicit user opt-in and a UI and so on, and additional unmaintained packages should not be removed during the lifetime of a Fedora release, I think that removing retired packages during upgrades should live in the upgrader, not be implemented as an RPM package (or at least not as an RPM package that's installed by default). The upgrade system is the only place that is in a position to actively prompt the user in a meaningful way to obtain explicit, informed opt-in consent to this.

(The lightweight version of this would be to require people to opt in in advance by installing the special fedora-retired-packages RPM. People who know enough to manually select and install the package can be presumed to know what they're doing and be making an informed choice to accept whatever package retirements Fedora wants to push.)

PS: I was going to consider this different from the existing situation with fedora-obsolete-packages for various hand-waving reasons, but the more I look at what packages Fedora has removed through the fedora-obsolete-packages RPM, the more I think that the two should be mostly merged together and treated very similarly (ie, require explicit opt-in). The current fedora-obsolete-packages goes well beyond merely removing packages that cause upgrade problems (unless you take a rather expansive view of 'upgrade problems').

FedoraRemovingMustBeOptIn written at 22:44:24; Add Comment

2020-06-17

How applications autostart on modern Linux desktops

A while back I mentioned that part of Microsoft Teams' misbehavior was autostarting when you logged in; recently, because I was testing some things on my laptop with alternate desktops, that behavior led to me uninstalling Teams. So all in all, it seemed like a good time to read up on how applications get automatically started when you log in (if they do) on modern Linux desktops like Gnome and Cinnamon.

(This is not normally an issue for me in my desktop environment, because everything in it is explicitly started by hand in a shell script.)

Unsurprisingly, there's a freedesktop.org standard on this, the Desktop Application Autostart Specification, which builds on the .desktop file specification. The simple version is that you set applications to autostart by installing an appropriate .desktop file for them into either /etc/xdg/autostart (for a system-wide autostart on login) or ~/.config/autostart (for an autostart for an individual user).

There are a number of special settings keys that can be in these .desktop files. First, you can have a OnlyShowIn or NotShowIn key that controls which desktops should autostart this or not autostart it (the specification misspells these keys in some mentions of them). Second, you can have a Hidden=true key, in which case the application should not autostart (in any desktop). For obvious reasons, the latter key is most useful in your personal autostart directory.

Some desktops have custom keys of their own, and even custom locations for additional .desktop files to autostart; for example KDE before KDE 5 apparently also used ~/.kde/Autostart (see here). An important and common property is X-GNOME-Autostart-enabled, which is (still) in wide use despite apparently being deprecated. In particular, Cinnamon appears to implement disabling of standard autostart things by copying their .desktop file to your ~/.config/autostart directory and adding a line to the end with 'X-GNOME-Autostart-enabled=false'.

(GNOME .desktop files can also have a phase of GNOME's startup that they happen in; see here and here. KDE .desktop files apparently have somewhat similar properties too.)

Some desktops have their own custom locations for various special things, or have had in the past (eg, also, and for LXDE). However, desktops don't necessarily use custom locations and settings. I know that with Cinnamon, if you add a new thing to be done on startup, Cinnamon puts a new .desktop file in your ~/.config/autostart.

More minimal 'desktops' may or may not automatically support .desktop autostarts. However, according to the Arch wiki's page on XDG Autostart, there are standalone programs that will do this for you if you want them to. On my normal machines, my own window manager environment is so divergent that I don't think autostarting .desktop files is of any use to me, so I'm not planning to try any of them.

(My work laptop runs a more or less standard Cinnamon environment, which automatically handles autostarting things. I believe that Cinnamon considers itself a form of GNOME for OnlyShowIn and NotShowIn and so on .desktop keys. Cinnamon certainly disables autostarted things using a GNOME specific key.)

Applications can arrange to autostart in at least two ways, the honest way and the sneaky way. The honest way is to put a copy of their .desktop file into /etc/xdg/autostart. The sneaky way is to wait until you run them once, then copy their .desktop file into your ~/.config/autostart directory (re-copying this file every time they're run is optional). Based on poking through the RPM package for Microsoft Teams (and also how they apparently have a preferences setting about this), Teams appears to do this the sneaky way.

DesktopAppAutostart written at 22:11:48; Add Comment

A scrolling puzzle involving GTK+, XInput, and alternate desktops (on Fedora)

Recently I discovered that cawbird wasn't recognizing certain 'scroll up' events in parts of its interface. This wasn't a new issue, although I originally thought it was; instead for years I'd been missing some of cawbird's functionality without noticing (and before it, corebird). I don't know exactly what the core problem is, but part of it appears to be some sort of interaction between desktop environments (or the lack of them) and the new approach to handling input devices using the X Input Extension.

The actual Cawbird issue is somewhat complicated and tangled, but fortunately it can be boiled down to a simpler situation through a test program that prints GTK mouse scroll event information (a copy of my version is here). For background, when vertical scrolling happens in GDK, you can see either or both of specific smooth scrolling events, with a Y delta of some value, and 'scroll up' and 'scroll down' events, which appear to always have a Y delta of 0.

On my desktop running fvwm outside of a desktop environment like Gnome, what I see from the test program when I use my mouse scroll wheel is just a stream of scroll up and scroll down events from a source device of 'Core Pointer'. On my work laptop running Cinnamon, scrolling on the touchpad generates smooth scrolling events with various Y deltas depending on how fast I'm moving my fingers, while using the scroll wheel on an external mouse generates both a smooth scrolling event (with a Y delta of -1 or +1 depending on the direction) and a 'scroll up' (or 'scroll down') event; these events have a source device of either the touchpad or the USB mouse, although xinput says that there is an overall 'Core Pointer' device.

As far as I can tell, xinput and the X servers are reporting that the mice involved are set up the same way; the physical mouse (and the touchpad) are extended input devices handled by XINPUT. But something about my fvwm environment or how I start X on my desktop is causing these GTK smooth scroll events to not be generated, to the confusion of at least some programs (and there will probably be more in the future). Unfortunately I have no ideas about what it might be or how to track it down.

(After some further testing, I can say that OpenBox on my laptop and Cinnamon inside a VMWare virtual machine both cause GTK to generate smooth scroll events. The VMWare virtual machine is using my desktop's mouse, but the xinput mouse configuration is different because of VMWware stuff.)

XInputGtkScrollPuzzle written at 00:37:08; Add Comment

2020-06-13

An interesting combination of flaws in some /etc/mailcap handling

Somewhat recently we ran into an interestingly tangled issue around /etc/mailcap and MIME handlers on our Ubuntu 18.04 user login machines, one of those situations where there seem to be multiple problems that when combined together lead to an undesirable result. What happened is that we installed the docx2txt Ubuntu package after it was requested by someone, but then found that this broke at least exmh's ability to display MS Office 'docx' file attachments. However, the interesting story is why.

As part of its package, docx2txt includes a /usr/lib/mime/packages file to describe what it can be used to display, which then causes update-mime to update the MIME handling information in /etc/mailcap. Because docx2txt prints what it converts to standard output, its mailcap entry has the 'copiousoutput' tag, and also appears to set its priority to 2, which is relatively low (5 is the default). The first thing that goes wrong is that docx2txt has an uncaught typo in this; it actually sets 'prority' to 2, leaving it at the default priority of 5. Also installed on our Ubuntu machines is LibreOffice, and LibreOffice Writer also has /usr/lib/mime/packages file. LibreOffice's entry for docx files has priority 3, theoretically higher than docx2txt's (and a standard condition to say 'I need to be in a GUI session to work'), but docx2txt's typo means that docx2txt's mailcap entry should actually be preferred over LibreOffice's.

The second thing that happens, which is at least unclear, is that update-mime doesn't pass the priority field through to /etc/mailcap. I think update-mime orders the generated /etc/mailcap from highest priority to lowest, and assumes that programs that use mailcap will pick the first matching entry. If this is what you're supposed to do to handle priorities in mailcap entries, I couldn't find anything that explicitly said it. Since this ordering doesn't seem to be explicitly written up, it's at most folk knowledge and you have to hope that the mailcap parser used by any particular program follows this. Update-mime also doesn't reject docx2txt's partially malformed mailcap line; instead it accepts it as an entry with the default priority (and puts the 'prority' field in the generated /etc/mailcap, where it may mislead you if you're reading fast).

The third thing going wrong is that exmh turns out to have bad handling of mailcap entries that write their results to standard output, so that you can theoretically display it inline. What you would expect to happen is that exmh would run the handler (either automatically or on request) and then display the result inline. Instead, it has a little display for that attachment that looks like you can't do anything (normally it will say 'you can view this with ...', so you know the section can be handled), and if you actually ask exmh to run the mailcap handler to generate the output, it writes the generated output to its standard error (which almost certainly isn't connected to anything useful). Given that this is spectacularly useless, exmh clearly hasn't been used very much with mailcap entries that want to do this instead of running an external program that will display things on its own.

Exmh's bad handling of 'copiousoutput' mailcap entries wouldn't be an issue except for the mangled priority field of docx2txt; without that, exmh picks LibreOffice instead (which works fine). Docx2txt's bad 'prority' field wouldn't have persisted if update-mime (or some other tool) checked for and rejected improperly formed mailcap entries; instead update-mime covered up the problem and increased docx2txt's priority over what it had been intended. It took a cascade of flaws to expose this issue.

(Our solution was to uninstall docx2txt again. It's not important enough to break exmh for people using it, and anything else that may also have problems with such an entry. Now that I understand the issue, I will see if I have enough energy to file a Debian bug report against docx2txt, which still has the bug in the current package source. Of course it will be years before any bug fix is available in Ubuntu.)

MailcapDocx2txtTangle written at 23:31:28; Add Comment

2020-06-09

My mixed feelings about 'swap on zram' for Linux

Recently I read about how Fedora will be enabling 'swap on zram', including for upgraded machines, in a future version of Fedora. I suspect that a similar change may some day come to Ubuntu as well, because it's an attractive feature from some perspectives. My feelings are a bit more mixed.

Zram is a dynamically sized compressed block device in RAM (ie a compressed ramdisk); 'swap on zram' is using a zram device as a swap device (or as your sole swap device). This effectively turns inactive RAM pages into compressed RAM in an indirect way while pacifying the kernel's traditional desire to have some swap space. The pitch for swap on zram is very nicely summarized on the Fedora page as 'swap is useful, except when it's slow'. Being in RAM, swap on zram is very fast; it's the fastest swap device you can have, faster than SSD or even NVMe.

(This implies that how much of an advantage swap on zram is for your system depends partly on how fast your existing swap storage is. But RAM is still much faster than even NVMe.)

The drawback of swap on zram is that it is not really freeing up all of your memory to 'swap things out'; instead the estimate is that it will generally compress to about half the previous size. This drawback is the source of my mixed feelings about swap on zram for my Fedora desktops and our Ubuntu servers.

On my Fedora desktops, I generally barely use any swap space, which means that swap on zram would be harmless. If I do temporarily use a surge of swap space, being able to get the contents back fast is probably good; Linux has generally had an irritating tendency to swap out things I wanted, like bits of my window manager's processes. Both my home machine and my work machine have 32 GB of RAM, and peak swap usage over the past 120 days has been under a gigabyte, so I'm barely going to notice the memory effects. As a result I'm likely to leave swap on zram in its default enabled state when Fedora gives it to me.

Unfortunately this is not the case for our Ubuntu LTS servers. Those of our Ubuntu servers that use much swap at all tend to eventually end up with their swap space full or mostly full of completely idle data that just sits there. Keeping even a compressed version of this data in RAM is not what we want; we really want it to be swapped out of memory entirely. Swap on zram would be a loss of RAM for us on our Ubuntu servers. As a result, if and when Ubuntu enables this by default, I expect us to turn it off again.

One way to put this is that swap on zram is faster than conventional swap but not as useful and effective for clearing RAM. Which of these is more important is not absolute but depends on your situation. If you're actively swapping, then speed matters (fast swap lowers the chances of swapping yourself to death). If you're instead pushing out idle or outright dormant memory in order to make room for more productive uses of that RAM, then clearing RAM matters most.

SwapOnZramMixedFeelings written at 00:02:36; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.