2017-06-23
In praise of uBlock Origin's new 'element zapper' feature
The latest versions of uBlock Origin have added a new feature, the element zapper. To quote the documentation:
The purpose of the element zapper is to quickly deal with the removal of nuisance elements on a page without having to create one or more filters.
uBlock Origin has always allowed you to permanently block page elements, and a while back I started using it aggressively to deal with the annoyances of modern websites. This is fine and works nicely, but it takes work. I have to carefully pick out what I want to target, maybe edit the CSS selector uBlock Origin has found, preview what I'm actually going to be blocking, and then I have a new permanent rule cluttering up my filters (and probably slightly growing Firefox's memory usage). This work is worth it for things that I'm going to visit regularly, but some combination of the amount of work required and the fact that I'd be picking up a new permanent rule made me not do it for pages I was basically just visiting once. And usually things weren't all that annoying.
Enter Medium and their obnoxious floating sharing bar at the
bottom of pages.
These things can be blocked on Medium's website itself with a
straightforward rule, but the problem is that tons of people use
Medium with custom domains. For example, this article
that I linked to in a recent entry. These days it seems like
every fourth article I read is on some Medium-based site (I exaggerate,
but), and each of them have the Medium sharing bar, and each of
them needs a new site-specific blocking rule unless I want to
globally block all <divs> with the class js-stickyFooter
(until
Medium changes the name).
(Globally blocking such a <div> is getting really tempting, though. Medium feels like a plague at this point.)
The element zapper feature deals with this with no fuss or muss. If I wind up reading something on yet another site that's using Medium and has their floating bar, I can zap it away in seconds The same is true of any number of floating annoyances. And if I made a mistake and my zapping isn't doing what I want, it's easy to fix; since these are one-shot rules, I can just reload the page to start over from scratch. This has already started encouraging me to do away with even more things than before, and just like when I started blocking elements, I feel much happier when I'm reading the resulting pages.
(Going all the way to using Firefox's Reader mode is usually too much of a blunt hammer for most sites, and often I don't care quite that much.)
PS: Now that I think about it, I probably should switch all of my
per-site blocks for Medium's floating bar over to a single
'##div.js-stickyFooter
' block. It's unlikely to cause any collateral
damage and I suspect it would actually be more memory and CPU
efficient.
(And I should probably check over my personal block rules in general, although I don't have too many of them.)
My situation with Twitter and my Firefox setup (in which I blame pseudo-XHTML)
Although it is now a little bit awkward to do this, let's start with my tweet:
I see Twitter has broken viewing regular Tweets in a browser that doesn't run JavaScript (gives endless redirections to the mobile site).
Twitter does this with a <noscript> meta-refresh, for example:
<noscript><meta http-equiv="refresh" content="0; URL=https://mobile.twitter.com/i/nojs_router?path=%2Fthatcks%2Fstatus%2F877738130656313344"></noscript>
Since I have JavaScript forced off for almost everyone in my main
Firefox (via NoScript), Twitter
included, my Firefox acts on this <noscript>
block. What is
supposed to happen here is that you wind up on the mobile version
of the tweet, eg, and
then just sit there with things behaving normally. In my development
tree Firefox, the version of
this page that I get also contains another <noscript> meta-refresh:
<noscript><meta content="0; URL=https://mobile.twitter.com/i/nojs_router?path=%2Fthatcks%2Fstatus%2F877738130656313344" http-equiv="refresh" /></noscript>
This is the same URL as the initial meta-refresh, and so Firefox sits there going through this cycle over and over and over again, and in the mean time I see no content at all, not even the mobile version of the tweet.
In other environments, such as Fedora 25's system version of Firefox 54, Lynx, and wget, the mobile version of the tweet is a page without the circular meta-refresh. At first this difference mystified me, but then I paid close attention to the initial HTML I was seeing in the page source. Here is the start of the broken version:
<!DOCTYPE html> <html dir="ltr" lang="en"> <meta charset="utf-8" /> <meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0" /> <noscript>[...]
(I suspect that this is HTML5.)
And here is the start of the working version:
<?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.1//EN" "http://www.openmobilealliance.org/tech/DTD/xhtml-mobile11.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> [... much more verbiage ...]
Although this claims to be some form of XHTML in its declarations,
Twitter is serving this with a Content-Type of text/html
, which
makes it plain old HTML soup as far as Firefox is concerned (which
is a famous XHTML issue).
What I don't understand is why Twitter serves HTML5 to me in one
browser and pseudo-XHTML to me in another. As far as I can tell,
the only significant thing that differs here between the system
version of Firefox and my custom-compiled one is the User-Agent
(and in particular both are willing to accept XHTML). I can get
Twitter to serve me HTML5 using wget
, but it happens using
either User-Agent string:
wcat
--user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0' https://mobile.twitter.com/thatcks/status/877738130656313344 | less
I assume that one of the issues here is that when Twitter decided to start forcing non-JavaScript browsers to the mobile version of tweets, they forgot to update the HTML5 version of the mobile tweet page to not have its own forcing of non-JavaScript people to what was originally (presumably) somewhere else. I suspect that this is because the HTML5 version is something very few people actually get, so the Twitter developers just forgot that it existed.
(Both versions of the mobile page try to load some JavaScript, but the HTML5 version seems to have more of it.)
Sidebar: How I worked around this
Initially I went on a long quest to try to find an extension that would turn this off or some magic trick that would make Firefox ignore it (and I failed). It turns out that what I need is already built into NoScript; the Advanced settings have an option for 'Forbid META redirections inside <NOSCRIPT> elements', which turns off exactly the source of my problems. This applies to all websites, which is a bit broader of a brush than would be ideal, but I'll live with it for now.
(I may find out that this setting breaks other websites that I use, although I hope not.)
2017-06-16
The (current) state of Firefox Nightly and old extensions
Back in January in my entry on how ready my Firefox extensions are for Firefox Electrolysis, I said that Firefox's release calendar suggested that Firefox's development version (aka 'Nightly') would stop supporting old non-Electrolysis extensions some time around June or July. It's now mid June and some things have happened, but I'm not sure where Mozilla's timeline is on this. So here is what I know.
At the start of May, Firefox Nightly landed bug 1352204, which is
about disabling a lot of older extensions on Nightly. Mozilla has
an information page about this in their wiki, and various news
outlets noticed and reported on this change shortly after it went
live, which means I'm late to the party here. As the Mozilla page covers, you can fix
this by setting the about:config
option
extensions.allow-non-mpc-extensions
to true
. I've done this
ever since I found the option and everything appears to still work
fine in the current Nightly.
(I had some weird things happen with Youtube that caused me to not update my Firefox build for a month or so because I didn't want to deal with tracking the issue down, but when I started to test more extensively they went away. Problems that vanish on their own can be the best problems.)
This change itself doesn't seem to be how Mozilla intends to turn off old extensions, theoretically in Firefox 57. That seems to be bug 1336576, expanded in a Mozilla wiki entry. Based on the Mozilla wiki entry, it appears that Firefox's development code base (and thus Nightly) will continue to allow you to load old extensions even after Firefox 57 is released provided that you flip a magic preference. Firefox 57 itself will not allow you to do so; the preference will apparently do nothing.
As long as Mozilla has legacy extensions that they care about, I believe that the actual code to load and operate such extensions will be present and working in the Firefox code base; this is the 'signed by Mozilla internally' case in their compatibility table. This implies that even if Mozilla disables the preference in the development version, you can force-override this with a code change if you build your own Firefox (which is what I do). You may not be able to turn Electrolysis on if you have such old legacy extensions, but presumably your addons are more important than Electrolysis (this is certainly the case for me).
All of this makes me much happier about the state of my personal Firefox than I used to be, because it looks like the point where many of my current extensions will fall over is much further away than I thought it was. Far from being this summer, it may be next summer, or evn further away than that, and perhaps by then the release of Firefox 57+ will have caused more of the addons that I care about to be updated.
(However, not all of the omens for updated addons are good. For example, Self-Destructing Cookies now explicitly marks itself as incompatible with Electrolysis because apparently addon can't monitor sites' LocalStorage usage in e10s. This suggests that there are important gaps in what addons can now do, gaps that Mozilla may or may not close over time. At least this particular case is a known issue, though; see bugs 1333050, 1329745, and 1340511 (via the addons page for Cookie Autodelete, which I was recently pointed at by a helpful reader of Wandering Thoughts).)
2017-06-05
Another case of someone being too clever in their User-Agent
field
Every so often, something prompts me to look at the server logs for
Wandering Thoughts in some detail to see what things are
lurking under the rocks. One area I wind up looking at is what
User-Agent
s are fetching my syndication feeds; often interesting
things pop out (by which I mean things that make me block people). In a recent case, I happened to
spot the following User-Agent
:
Mozilla/5.0 (compatible) AppleWebKit Chrome Safari
That's clearly bogus, in a way that smells of programming by
superstition. Someone
has heard that mentioning other user-agents in your User-Agent
string is a good idea, but they don't quite understand the reason
why or the format that people use. So instead of something that
looks valid, they've sprayed in a random assortment of browser
and library names.
As with the first too-clever User-Agent
, my initial reaction was to block
this user agent entirely. It didn't help that it was coming from
random IPs and making no attempt to use conditional GET
. After running this way for a few days and
seeing the fetch attempts continue, I got curious enough to do an
Internet search for this exact string to see if I could turn up
someone who'd identified what particular spider this was.
I didn't find that. Instead, I found the source code for this,
which comes from Flym, an Android feed reader (or maybe this fork of it). So, contrary to how this
User-Agent
makes it look, this is actually a legitimate feed
reader (or as legitimate a feed reader as it can be if it doesn't
do conditional GET
, which is another debate entirely). Once
I found this out, I removed my block of it, so however many people
who are using Flym and spaRSS can now read my feed again.
(Flym is apparently based on Sparse-RSS, but the current version of
that sends a User-Agent
of just "Mozilla/5.0"
(in here),
which looks a lot less shady because it's a lot more generic. Claiming
to be just 'Mozilla/5.0
' is the 'I'm not even trying' of User-Agents
.
Interestingly, I do appear to have a number of people pulling Wandering
Thoughts feeds with this User-Agent
, but it's so generic that I have
no idea if they're using Sparse-RSS or something else.)
In the past I've filed bugs against open
source projects over this sort of issue, but sadly Flym doesn't appear to accept bug
reports through Github and at the moment I don't feel energetic
enough to even consider something more than that. I admit that
part of it is the lack of conditional GET
; if you don't
put that into your feed reader, I have to assume that you don't
care too much about HTTP issues in general.
(See my views on what your User-Agent
header should include and
why. Flym, spaRSS, and Sparse-RSS all fall
into the 'user agent' case, since they're used by individual users.)
PS: Mobile clients should really, really support conditional GET
,
because mobile users often pay for bandwidth (either explicitly or
through monthly bandwidth limits) and conditional GET
on feeds
holds out the potential of significantly reducing it. Especially
for places with big feeds, like Wandering Thoughts. But this
is not my problem.