Wandering Thoughts archives


In praise of uBlock Origin's new 'element zapper' feature

The latest versions of uBlock Origin have added a new feature, the element zapper. To quote the documentation:

The purpose of the element zapper is to quickly deal with the removal of nuisance elements on a page without having to create one or more filters.

uBlock Origin has always allowed you to permanently block page elements, and a while back I started using it aggressively to deal with the annoyances of modern websites. This is fine and works nicely, but it takes work. I have to carefully pick out what I want to target, maybe edit the CSS selector uBlock Origin has found, preview what I'm actually going to be blocking, and then I have a new permanent rule cluttering up my filters (and probably slightly growing Firefox's memory usage). This work is worth it for things that I'm going to visit regularly, but some combination of the amount of work required and the fact that I'd be picking up a new permanent rule made me not do it for pages I was basically just visiting once. And usually things weren't all that annoying.

Enter Medium and their obnoxious floating sharing bar at the bottom of pages. These things can be blocked on Medium's website itself with a straightforward rule, but the problem is that tons of people use Medium with custom domains. For example, this article that I linked to in a recent entry. These days it seems like every fourth article I read is on some Medium-based site (I exaggerate, but), and each of them have the Medium sharing bar, and each of them needs a new site-specific blocking rule unless I want to globally block all <divs> with the class js-stickyFooter (until Medium changes the name).

(Globally blocking such a <div> is getting really tempting, though. Medium feels like a plague at this point.)

The element zapper feature deals with this with no fuss or muss. If I wind up reading something on yet another site that's using Medium and has their floating bar, I can zap it away in seconds The same is true of any number of floating annoyances. And if I made a mistake and my zapping isn't doing what I want, it's easy to fix; since these are one-shot rules, I can just reload the page to start over from scratch. This has already started encouraging me to do away with even more things than before, and just like when I started blocking elements, I feel much happier when I'm reading the resulting pages.

(Going all the way to using Firefox's Reader mode is usually too much of a blunt hammer for most sites, and often I don't care quite that much.)

PS: Now that I think about it, I probably should switch all of my per-site blocks for Medium's floating bar over to a single '##div.js-stickyFooter' block. It's unlikely to cause any collateral damage and I suspect it would actually be more memory and CPU efficient.

(And I should probably check over my personal block rules in general, although I don't have too many of them.)

UBlockOriginZapperPraise written at 23:16:08; Add Comment

My situation with Twitter and my Firefox setup (in which I blame pseudo-XHTML)

Although it is now a little bit awkward to do this, let's start with my tweet:

I see Twitter has broken viewing regular Tweets in a browser that doesn't run JavaScript (gives endless redirections to the mobile site).

Twitter does this with a <noscript> meta-refresh, for example:

<noscript><meta http-equiv="refresh" content="0; URL=https://mobile.twitter.com/i/nojs_router?path=%2Fthatcks%2Fstatus%2F877738130656313344"></noscript>

Since I have JavaScript forced off for almost everyone in my main Firefox (via NoScript), Twitter included, my Firefox acts on this <noscript> block. What is supposed to happen here is that you wind up on the mobile version of the tweet, eg, and then just sit there with things behaving normally. In my development tree Firefox, the version of this page that I get also contains another <noscript> meta-refresh:

<noscript><meta content="0; URL=https://mobile.twitter.com/i/nojs_router?path=%2Fthatcks%2Fstatus%2F877738130656313344" http-equiv="refresh" /></noscript>

This is the same URL as the initial meta-refresh, and so Firefox sits there going through this cycle over and over and over again, and in the mean time I see no content at all, not even the mobile version of the tweet.

In other environments, such as Fedora 25's system version of Firefox 54, Lynx, and wget, the mobile version of the tweet is a page without the circular meta-refresh. At first this difference mystified me, but then I paid close attention to the initial HTML I was seeing in the page source. Here is the start of the broken version:

<!DOCTYPE html>
<html dir="ltr" lang="en">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0" />

(I suspect that this is HTML5.)

And here is the start of the working version:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.1//EN" "http://www.openmobilealliance.org/tech/DTD/xhtml-mobile11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  [... much more verbiage ...]

Although this claims to be some form of XHTML in its declarations, Twitter is serving this with a Content-Type of text/html, which makes it plain old HTML soup as far as Firefox is concerned (which is a famous XHTML issue).

What I don't understand is why Twitter serves HTML5 to me in one browser and pseudo-XHTML to me in another. As far as I can tell, the only significant thing that differs here between the system version of Firefox and my custom-compiled one is the User-Agent (and in particular both are willing to accept XHTML). I can get Twitter to serve me HTML5 using wget, but it happens using either User-Agent string:

wcat --user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0' https://mobile.twitter.com/thatcks/status/877738130656313344 | less

I assume that one of the issues here is that when Twitter decided to start forcing non-JavaScript browsers to the mobile version of tweets, they forgot to update the HTML5 version of the mobile tweet page to not have its own forcing of non-JavaScript people to what was originally (presumably) somewhere else. I suspect that this is because the HTML5 version is something very few people actually get, so the Twitter developers just forgot that it existed.

(Both versions of the mobile page try to load some JavaScript, but the HTML5 version seems to have more of it.)

Sidebar: How I worked around this

Initially I went on a long quest to try to find an extension that would turn this off or some magic trick that would make Firefox ignore it (and I failed). It turns out that what I need is already built into NoScript; the Advanced settings have an option for 'Forbid META redirections inside <NOSCRIPT> elements', which turns off exactly the source of my problems. This applies to all websites, which is a bit broader of a brush than would be ideal, but I'll live with it for now.

(I may find out that this setting breaks other websites that I use, although I hope not.)

TwitterFirefoxMetarefresh written at 00:25:59; Add Comment


The (current) state of Firefox Nightly and old extensions

Back in January in my entry on how ready my Firefox extensions are for Firefox Electrolysis, I said that Firefox's release calendar suggested that Firefox's development version (aka 'Nightly') would stop supporting old non-Electrolysis extensions some time around June or July. It's now mid June and some things have happened, but I'm not sure where Mozilla's timeline is on this. So here is what I know.

At the start of May, Firefox Nightly landed bug 1352204, which is about disabling a lot of older extensions on Nightly. Mozilla has an information page about this in their wiki, and various news outlets noticed and reported on this change shortly after it went live, which means I'm late to the party here. As the Mozilla page covers, you can fix this by setting the about:config option extensions.allow-non-mpc-extensions to true. I've done this ever since I found the option and everything appears to still work fine in the current Nightly.

(I had some weird things happen with Youtube that caused me to not update my Firefox build for a month or so because I didn't want to deal with tracking the issue down, but when I started to test more extensively they went away. Problems that vanish on their own can be the best problems.)

This change itself doesn't seem to be how Mozilla intends to turn off old extensions, theoretically in Firefox 57. That seems to be bug 1336576, expanded in a Mozilla wiki entry. Based on the Mozilla wiki entry, it appears that Firefox's development code base (and thus Nightly) will continue to allow you to load old extensions even after Firefox 57 is released provided that you flip a magic preference. Firefox 57 itself will not allow you to do so; the preference will apparently do nothing.

As long as Mozilla has legacy extensions that they care about, I believe that the actual code to load and operate such extensions will be present and working in the Firefox code base; this is the 'signed by Mozilla internally' case in their compatibility table. This implies that even if Mozilla disables the preference in the development version, you can force-override this with a code change if you build your own Firefox (which is what I do). You may not be able to turn Electrolysis on if you have such old legacy extensions, but presumably your addons are more important than Electrolysis (this is certainly the case for me).

All of this makes me much happier about the state of my personal Firefox than I used to be, because it looks like the point where many of my current extensions will fall over is much further away than I thought it was. Far from being this summer, it may be next summer, or evn further away than that, and perhaps by then the release of Firefox 57+ will have caused more of the addons that I care about to be updated.

(However, not all of the omens for updated addons are good. For example, Self-Destructing Cookies now explicitly marks itself as incompatible with Electrolysis because apparently addon can't monitor sites' LocalStorage usage in e10s. This suggests that there are important gaps in what addons can now do, gaps that Mozilla may or may not close over time. At least this particular case is a known issue, though; see bugs 1333050, 1329745, and 1340511 (via the addons page for Cookie Autodelete, which I was recently pointed at by a helpful reader of Wandering Thoughts).)

FirefoxElectrolysisOldExtensions written at 01:35:49; Add Comment


Another case of someone being too clever in their User-Agent field

Every so often, something prompts me to look at the server logs for Wandering Thoughts in some detail to see what things are lurking under the rocks. One area I wind up looking at is what User-Agents are fetching my syndication feeds; often interesting things pop out (by which I mean things that make me block people). In a recent case, I happened to spot the following User-Agent:

Mozilla/5.0 (compatible) AppleWebKit Chrome Safari

That's clearly bogus, in a way that smells of programming by superstition. Someone has heard that mentioning other user-agents in your User-Agent string is a good idea, but they don't quite understand the reason why or the format that people use. So instead of something that looks valid, they've sprayed in a random assortment of browser and library names.

As with the first too-clever User-Agent, my initial reaction was to block this user agent entirely. It didn't help that it was coming from random IPs and making no attempt to use conditional GET. After running this way for a few days and seeing the fetch attempts continue, I got curious enough to do an Internet search for this exact string to see if I could turn up someone who'd identified what particular spider this was.

I didn't find that. Instead, I found the source code for this, which comes from Flym, an Android feed reader (or maybe this fork of it). So, contrary to how this User-Agent makes it look, this is actually a legitimate feed reader (or as legitimate a feed reader as it can be if it doesn't do conditional GET, which is another debate entirely). Once I found this out, I removed my block of it, so however many people who are using Flym and spaRSS can now read my feed again.

(Flym is apparently based on Sparse-RSS, but the current version of that sends a User-Agent of just "Mozilla/5.0" (in here), which looks a lot less shady because it's a lot more generic. Claiming to be just 'Mozilla/5.0' is the 'I'm not even trying' of User-Agents. Interestingly, I do appear to have a number of people pulling Wandering Thoughts feeds with this User-Agent, but it's so generic that I have no idea if they're using Sparse-RSS or something else.)

In the past I've filed bugs against open source projects over this sort of issue, but sadly Flym doesn't appear to accept bug reports through Github and at the moment I don't feel energetic enough to even consider something more than that. I admit that part of it is the lack of conditional GET; if you don't put that into your feed reader, I have to assume that you don't care too much about HTTP issues in general.

(See my views on what your User-Agent header should include and why. Flym, spaRSS, and Sparse-RSS all fall into the 'user agent' case, since they're used by individual users.)

PS: Mobile clients should really, really support conditional GET, because mobile users often pay for bandwidth (either explicitly or through monthly bandwidth limits) and conditional GET on feeds holds out the potential of significantly reducing it. Especially for places with big feeds, like Wandering Thoughts. But this is not my problem.

UserAgentMoreTooClever written at 01:23:34; Add Comment

By day for June 2017: 5 16 23; before June; after June.

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.