Wandering Thoughts

Improving my web reading with Martin Tournoij's "readable" Firefox bookmarklet

Not that long ago, I set up Martin Tournoij's "fixed" bookmarklet to deal with CSS fixed elements. When I did this, I also decided to install Tournoij's "readable" bookmarklet, because it was right there and it felt potentially useful. With it sitting in my tab bar, I started trying it out on sites that I found not so readable, or even vaguely marginally non-readable, and to my surprise it's been a major quality of life improvement on many sites. I've become quite glad that I made it conveniently available.

What the "readable" bookmarklet does is it goes through every <p>, <li>, and <div> to force the text colour, size, weight, line spacing, and font family to reasonable values. It doesn't try to set the background colour, but it turns out that a lot of sites use a basically white background, so forcing the text colour is sufficient. All of this sounds very basic, but the result can be really marvelous. It's especially impressive on sites that don't feel as if they have obviously terrible text, just text that's a bit annoying. It turns out that what feels 'a bit annoying' to me is often harder to read than I was consciously aware of.

Why such simple restyling works so well in practice is somewhat sad. It turns out that a lot of sites make terrible text styling choices for clear readability. The obvious case is too-small text, but beyond that a lot of sites turn out to set a lower-contrast text colour, such as some shade of grey, unusually thin text through either weight or font choice, or both at once. Undoubtedly they think that the result looks good and is perfectly readable, but increasingly my eyes disagree with them.

Because I looked it up, here is specifically what is being set by the current bookmarklet. Currently, the "readable" bookmarklet runs the following Javascript:

javascript:(function() {
    document.querySelectorAll('p, li, div').forEach(function(n) {
        n.style.color = '#000';
        n.style.font = '500 16px/1.7em sans-serif';
    });
})();

The n.style.color is simple; #000 is black. The n.style.font is a little bit more complex, because it's using the shorthand font property in a specific format. This format sets the font-weight to '500', which is just a little bit bolder than normal ('400' is normal), the font-size to 16 px (which these days is a device-independent thing), the line-height to 1.7 em for a pretty generous spacing between lines, and the font-family to your general sans-serif font. People who prefer serif fonts may want to change that to 'serif', and in general now that I look at it you might want to tinker with the 16px and the line spacing as well, depending on your preferences.

(My standard Firefox font is set to the default Fedora 'serif' font, currently DejaVu Serif according to Firefox, at size '15'. I could probably reasonably change the '16px/1.7em sans-serif' in the bookmarklet to '15px/1.5em serif' or so, but at the moment I don't feel inclined to do so; if I'm irritated enough to poke the bookmarklet, I might as well make the page really readable.)

Redirecting paths that start with two slashes in Apache

Suppose that you have a dynamic web application of some sort that sits behind Apache, and an URL for one of your application's pages where the path starts with two slashes instead of one starts going around. In other words, people are sharing 'https://example.org//app/page', instead of the version with one slash at the start of the path. You can support this in your web app (with some potential cautions), but you would prefer to have Apache redirect the non-ideal URL to the canonical URL.

Under normal circumstances, this sort of selective redirection should be straightforward using mod_rewrite. If you wanted to rewrite only a single specific bad path instead of all potential ones, I'd expect something like the following to work:

RewriteCond %{REQUEST_URI} "=//app/page"
RewriteRule ^.* https://example.org/app/page

(This is one of the cases where exact string matching in RewriteCond is a useful thing.)

However, this appears not to work in at least a .htaccess file, and I suspect it won't work in the Apache configuration file either. Although Apache's %{REQUEST_URI} gives you the full URL path even in a .htaccess, it appears that Apache canonicalizes it for the purposes of things like RewriteCond and so turns the two leading slashes into one. This canonicalization isn't passed through to CGIs, though; they will see the original "//app/page" version.

(This canonicalization appears to apply to any / in the URL path, not just the first ones. If you write a condition for "/app/dir/page", it will match for URLs with any amount of additional slashes, eg "//app////dir///page" will match.)

Instead, the only way I found to do this was with Apache's special %{THE_REQUEST} variable for the full HTTP request line. As the mod_rewrite documentation covers, this value has not been escaped, which may cause you heartburn if people get clever or you want to do general matching. So the rule I wound up with looks like:

RewriteCond %{THE_REQUEST} "^GET //app/page "
RewriteRule ^.* https://example.org/app/page

This match is very specific, since we're doing a very specific HTTP redirection. You'd want to be more complicated if you need to handle query variables, for example. But it works, unlike the other option.

Possibly I'm missing a clever trick that enables a better version of this. I don't really like matching things so specifically, but it seems to be what you have to reach for in this unusual situation.

Some brief notes on turning Firefox bookmarklets into convenient buttons

Yesterday when I talked about dealing with CSS fixed elements in Firefox I mentioned that one of the good looking solutions was Martin Tournoij's bookmarklet but that I lacked a good way to get access to bookmarklets because I don't use Firefox bookmarks and so don't have the bookmarks toolbar displayed. Today, after various poking around, I found out how to do this in my setup, which was partly difficult because Firefox sometimes hides one of the title bar and URL bar customization options.

First, what appears on the Firefox Bookmarks toolbar is mostly what's in the special "Bookmarks Toolbar" folder of your bookmarks (Firefox will add an "Other Bookmarks" menu at the end). You can't rename this folder and make a new version with all new contents; instead you have to make a new folder (say "Original Bookmarks Toolbar") and then move your current Bookmarks Toolbar contents to it one by one. This new folder has to go in either "Other Bookmarks" or "Bookmarks Menu". Once you've emptied out the Bookmarks Toolbar special folder, you can add bookmarklets to it and have them be the only thing in the Bookmarks toolbar. This is not particularly useful by itself unless you want an entire line of precious vertical content to always be consumed.

To make bookmarklets into almost buttons, you first should give them very short names (single letters are good). Then, you can add the entire Bookmarks Toolbar to the titlebar area, but it may be a little tricky. You get access to title bar customization by right click → "Customize toolbar...", but the Bookmarks toolbar doesn't appear as a customization element if you have it set to "Never show". Instead you need to set it to "Only show in new tab", at which point it will show up at the top (above "Drag your favorite items into the toolbar or overflow menu"). Now you can drag it to the titlebar, which will make the "Bookmarks Toolbar" entries for your bookmarklets show up in the titlebar once you exit customization and are on regular pages.

The "Bookmarks Toolbar" entries show as both a little globe icon and their title (which is why single letters are good, since they take only a bit of space). As far as I know there's no way to make the globe icon go away, and so bookmarklet buttons always take a bit more space than regular customization icons.

This is probably obvious, but clicking on the bookmarklet will invoke it; it's not a toggle. If you want to revert the effects of a bookmarklet like Martin Tournoij's, you need to refresh the page. With much more effort you could probably write an invertible bookmarklet, but I'm lazy (and also I don't know either Javascript or the modern browser DOM APIs, or for that matter if there's any easy way to go from nicely written out Javascript to something encoded ready to be a bookmarklet).

PS: I am already enjoying my new 'f' bookmarklet to make fixed position elements into regular ones. It's so convenient to just zap all the irritating headers away with a click. Maybe someday it'll have problems with sites that set a strict CSP, but not so far. For sites I visit regularly, I'm going to keep using Stylus styles because then they're applied automatically on the site (assuming the site is one that doesn't change its HTML around all the time).

Dealing with CSS fixed position headers and footers in Firefox

I'll start with what I tweeted:

Does anyone know of a Firefox addon that finds CSS 'fixed' position elements and removes that to make them non-fixed elements? There are a number of addons that will hide fixed elements entirely, but I would rather have them remain visible but scroll offscreen as I scroll.

The Firefox addons I know for hiding CSS 'fixed' elements are Sticky Ducky - clean the fixed elements, which seems to offer multiple levels of dealing with them, Fixed width and CSS Annihilator, which has additional effects but can be used selectively, Hide Fixed Elements, and Unfixed! - Fixed Elements Remover. Additionally, on Twitter Martin Tournoij shared a bookmarklet for this, which could easily be changed to set 'position: inherit !important;' instead of its current 'absolute'.

The reason I'm interested in this is the usual one of the plague of fixed headers and footers. Until now, I've mostly been zapping them away temporarily or permanently with uBlock Origin. Since I'm now using Stylus to make visited links visible again, I've also used it to modify a few sites to de-fix these elements instead of just removing them, which sparked my interest in a more general solution. Always hiding fixed elements isn't what I want, since some of the navigation is actually useful; I just don't want them on screen taking up space all the time. If I want to navigate, I'm perfectly capable of scrolling up (or down, but headers are more common than footers).

In looking at various of the sites I use, it's clear that applying this universally would be a bad idea (especially universally hiding fixed position elements). Thus, the ideal addon would have a site list of some sort and only take effect on some sites, with a simple toggle for it (and perhaps an option to either de-fix or hide fixed elements for each site). Martin Tournoij's bookmarklet would be a great option if only there was a way to have convenient access to it in a Firefox setup where you have the bookmarks toolbar hidden. Unfortunately as far as I know you can't put a button or a label for individual bookmarklets on the title bar.

(I don't use bookmarks, preferring to have HTML files instead.)

The Bookmarklets context menu addon (Github), adds a sub-menu for bookmarklets to the page context menu (theoretically; I haven't tested how well it works in Firefox today). It would obviously provide me relatively convenient access to this sort of Javascript driven CSS rewrites, which Martin Tournoij demonstrates can be used to to improve site readability in several ways. Greasemonkey is not exactly a bookmarklet thing, but it is about running Javascript actions on the page, and there even appear to be a variety of public userscripts to do this sort of thing (eg 1, 2).

Apache directory indexes will notice and exclude blocked URLs in them

Today I learned about an interesting and nice little Apache feature. If you have Apache generating its own automatic index pages for filesystem directories, and you block access to some things in a directory with <Location> blocks, Apache's generated index won't include what you blocked. It's as if the object doesn't exist. This is what you want (since attempts to access those things will fail), but it's more than I expected.

That sounds abstract, so let me make it concrete. We have an old legacy FTP site, which we've recently made available as a HTTPS site because browsers are removing support for FTP. For historical reasons, this FTP site has some symbolic links that create recursive structures; for example, it has a public_html symlink in the root that points to '.' (the current directory). Unfortunately, web spiders just love recursive structures and will crawl through them incessantly, with ever lengthening URLs.

(Web spider operators will probably tell you that they don't like recursive link situations like this. I have to go by observed behavior, which is that any number of web spiders don't appear to notice that /public_html/ is exactly the same content as / and /public_html/public_html/ and so on.)

We don't want to remove the symbolic links from the actual directory tree that's the FTP site, for various reasons (maybe they're there for some necessary reason, or at least have become embedded in historical FTP URLs). But the HTTPS site is new and we can drop whatever URLs we want from it. So I did the obvious simple thing:

<Location "/public_html">
  Order deny,allow
  Deny from all
</Location>

When I was verifying that this worked, I noticed that the top level index page for the FTP site no longer showed any public_html entry. Testing showed that this happened for other entries in the directory as well, if I temporarily added them.

The mod_autoindex documentation suggests that this is a standard feature that does general permission checks, based on the documentation for the ShowForbidden option to the IndexOptions directive. However, I haven't tested this with more complex situations, such as <Directory> instead of <Location> or more complicated permissions.

Some notes on Firefox's media autoplay settings as of Firefox 89

Back in 2018 I wrote about Firefox's media autoplay settings as of Firefox 63. Unsurprisingly, things have changed in the Firefox world since then, and for once they've changed for the better, at least from my perspective. My interest is in basically never having video autoplay. And when I say basically never, I mean it. In particular, I don't want Youtube to autoplay the next video in a playlist or the like.

The basic control for media autoplaying (for video and audio) is in Preferences → Privacy & Security, as covered in Allow or block media autoplay in Firefox. As mentioned there, if you want to block all videos you want 'block audio and video', not just 'block audio', which only blocks videos with un-muted audio and still allows silent ones to autoplay if they want to. This appears to affect everything, including bare video content such as direct links to .mp4s, which is a change from 2018. The limitation of this setting is that by itself, once you click to play a video in a tab, (auto)playing stays enabled until you close the tab again. So once you've let the first Youtube video play, Youtube (or anywhere else) will autoplay as much else as it wants to (in that tab). I prefer to specifically play each video.

To get more control, we need to turn to about:config and its collection of media.autoplay.* settings. Two resources are handy for this; the Firefox source code file AutoplayPolicy.cpp (which has been moved around in the Firefox source tree since 2018) and an additional commentary for media.autoplay.blocking_policy in StaticPrefList.yaml (via this Reddit question and answer).

The Privacy & Security setting controls media.autoplay.default. A setting of 0 means 'allow', 1 means 'block audio (including video with audio)', and 5 means 'block audio and video'. Once you've played video, the media.autoplay.blocking_policy preference controls what happens to future attempts to autoplay video in the same tab. A setting of 0 (the default) allows this to work, so Youtube gets to autoplay further videos. This is apparently called sticky activation. A setting of 1 is called transient activation, and apparently will only autoplay future videos within a narrow time window controlled by dom.user_activation.transient.timeout, which defaults to five seconds. Finally, a setting of 2 appears to always require user input, and it certainly stops Youtube autoplaying the next video for me. It's possible that a setting of 2 will break some sites, and you'll need '1' instead.

(This apparently replaces the old media.autoplay.enabled.user-gestures-needed setting. For a long discussion, see bug 1509933.)

Unfortunately, WebAudio seems to be another matter. Right now, in a normal Firefox the only way to disable WebAudio autoplaying seems to be to have media.autoplay.blocking_policy left at the default of 0 and media.autoplay.block-webaudio set to true. If you set another value for media.autoplay.blocking_policy, WebAudio can always autoplay if it wants to regardless of other settings. This may be something that Firefox would consider a bug. Fortunately WebAudio seems to be relatively rare on the web, and autoplaying WebAudio even more so.

(I base this on this demo, which autoplays unless I set media.autoplay.block-webaudio to true and media.autoplay.blocking_policy to 0, provided that I have Javascript enabled at all.)

The information about Firefox about:config settings in MDN's Autoplay guide for media and Web Audio APIs is somewhat outdated but might be useful, especially for media.autoplay.block-webaudio. Also, see this question and answer.

PS: A website that you've given camera or microphone permissions to is always allowed to autoplay video (or audio) regardless of your media autoplay settings. This hasn't changed since my entries in 2018 and Firefox will probably always continue to do this, since it's sensible for video conference tools.

PPS: In theory bug 1480281 might provide a way to get some information about Firefox's media autoplay decisions through either setting the MOZ_LOG environment variable or through about:networking's HTTP Logging. In practice I'm not sure if this works in release builds of Firefox. You'll probably need to read AutoplayPolicy.cpp to understand the messages.

The modern web design aesthetic of hiding visited links

Every since I started aggressively overriding websites so I could see what links I'd visited, it's become really clear to me that modern web design apparently hates making visited links look different and goes out of its way to not do so. This is most visible on the quite a lot of text focused websites that still stick to more or less the original web colours of black text on a white background with blue links. In the original basic web colour scheme, visited links would be a purple shade. But although all of these websites stick to the basic black, white, and blue colours, they wipe away the purple of visited links.

Some websites opt to style their links in some way other than with colours (often underlines), and for those websites not styling visited links differently is understandable (even though I disagree with it). These sites have decided to have text and text decorations in a single colour, and you can't significantly restyle visited links these days (for good reasons). So, for example, if you use a solid underline for unvisited links, you can't use a dashed one for visited ones (as far as I know). Other websites opt to significantly change some or many of the colours for whatever reasons, so not styling visited links differently avoids coming up with an additional colour (and colours are hard if you take it seriously).

(Also, if I'm understanding CSS right, if you set a colour for <a> elements it applies to them in all states. Once you start setting link colours, you have to go out of your way to colour visited, active, focused, or hovered links different than standard (unvisited) ones.)

However, quite a lot of text focused sites stick to the basic colours for text content but more or less deliberately wipe out any special colour for visited links. These sites could easily continue to let visited links be as they are when unstyled, but they don't. Whether this is done deliberately or is simply casually accepted, it's clearly part of today's web design aesthetic (and probably has been for a while).

I'm sure I noticed this subconsciously before, but actually creating site style after site style in Stylus has rubbed my nose in just how many of the sites I wanted to fix use the standard black, white, and blue colour scheme. It's also made me aware of how common a basic scheme of black, white, and underlined links is (it's probably the second most common one I alter).

The case of the very old If-Modified-Since HTTP header

Every so often I look at the top IP sources for Wandering Thoughts. Recently, I noticed that one relatively active IP was there because it was fetching my Atom syndication feed every few minutes, and on top of that it was always getting a HTTP 200 reply with the full feed. Usually my assumption is that these requests aren't using HTTP conditional GET at all, but I keep looking because I might find something like the Tiny Tiny RSS problem (which I can theoretically fix Tiny Tiny RSS). To my surprise, something a bit interesting is happening.

This feed fetcher was sending an If-Modified-Since HTTP header, but it had a rather striking value of 'Wed, 01 Jan 1800 00:00:00 GMT'. Naturally this doesn't match any Last-Modified value my feed has ever provided, and it wouldn't help if I used a time based comparison since all syndication feeds in the world have been changed since 1800.

Any time I see a very old timestamp like this, I suspect that there's code that has been handed an un-set zero value instead of an actual time (here, a Last-Modified time). Syndication feed fetchers are perhaps especially prone to this; they start out with no Last-Modified time when they fetch a feed for the first time, and then if they ever fail to parse a Last-Modified time properly they might write back an unset value. However, 1800 is a somewhat unusual zero value for time; I'm more used to Unix timestamps, where the zero value is January 1st 1970 GMT.

This feed fetcher identifies itself as 'NextCloud-News/1.0'. If that is this NextCloud application (also), it's written in PHP and is probably setting If-Modified-Since here using a PHP DateTime (or maybe it uses feed-io, I don't actually know PHP so I'm just grep'ing the codebase). I can't readily find any documentation on what the zero value for a DateTime is, or if it's even possible to wind up with one. Neither MySQL, PostgreSQL, nor SQLite appear to use 01 Jan 1800 as a zero value either. So on the whole I'm still lost.

(In passing I'll note that this user-agent value is not all that useful. To be useful, it should include the actual version number of the NextCloud-News release (they're up to 15.x, with 16.0.0 coming soon) and some URL for it, so I can be confident I have identified the right NextCloud News thing.)

PS: If this is a NextCloud-News code issue, correcting it would be nice (and please don't treat Last-Modified as a timestamp), but it would be better to use ETag and If-None-Match.

(This elaborates on a Twitter thread.)

Being able to see links I've visited in Firefox is startlingly better

I recently wrote about how I wanted to make links that I'd already visited be clearly visible in Firefox. At the time I was hoping for a general style that I could temporarily enable on anywhere and have work regardless of the site's choice of colours, link decorations, and other details. In writing the entry I discovered that Firefox supported the special colour name of 'VisitedText' for your standard visited link colour. While I considered general solutions, this led me to start writing some very quick site-specific styles with Stylus to use this on suitable sites. Each site style is the same:

a:visited {
  color: VisitedText !important;
}

There are a lot of sites that this very simple style works on, because it turns out that there are a lot of sites that are basically black text on white background; they've just decided to suppress visited links as a separate color (and sometimes fiddle with the link style too). At this point I've added Stylus styles for about a dozen sites.

(It turns out that Stylus has an interface for adding additional domains to a style so I theoretically don't need one style per site. In practice it's probably still easier to type this out anew for each new site I apply this to rather than find and add the domain name to an existing site. And this way, I can change the site-specific style if the site does something odd later.)

It's been startling how much better this has made my experience on these sites. It's encouraged my exploration of their links, has made re-visiting far less of an annoying experience, and in general has removed a source of friction that I didn't consciously realize existed. Each site is clearly nicer this way (and less quietly irritating). It at least feels as if the sites are more legible and easier to navigate.

(One of my guesses for why is that before, every time I considered opening a link I'd hover over it and try to remember if I'd already read it. Now I don't have to try that; the site tells me.)

On the one hand I feel happy that this has validated my effort. On the other hand, this vividly demonstrates that so many (other) sites are clearly damaging their own usability (at least for me) by deliberately hiding this information in the name of some design principle.

(Of course this fix isn't complete, because some sites also put their own tracking additions on their outbound links. This makes Firefox consider the full link to be 'not visited', even though I've already visited the link without the extra tracking goo. But that's life on the modern web.)

Firefox and the challenge of trying to make visited links clearly visible

I tweeted:

I wish I could work out a way in the Stylus Firefox addon to force-style visited links in something distinctive. In theory it's easy; in practice Firefox seems to sharply limit how you can style them these days for privacy, and my attempts so far aren't all that successful.

I frequently read articles on the Internet that have plenty of links to other interesting things, some of which I've read and some of which I haven't and would like to. On normal websites, this is what my very large browser history and the default visual distinction between visited and unvisited links is good for; it tells me what things I haven't read yet. Unfortunately, a steadily increasing number of websites style unvisited and visited links the same, defeating my attempts to see what I've already read. Sometimes they go the extra distance to style links so it's hard to even tell they are links.

With suitable effort, I could definitely use Stylus to create individual styles for every annoying site that made unvisited links have a distinctly different colour the way they're supposed to. But that's a lot of work. What I'd prefer to do is create a single global style, normally not on, that would make unvisited links (and perhaps visited ones) stand out no matter what colours and CSS styling the particular site was already using.

In the old days, this would have been pretty simple. There are a lot of text styles that aren't used much and stand out a lot, such as strike-throughs or heavy underlines. The result wouldn't necessarily look the best, but at least I could see what links I'd already read. Unfortunately, you can't do that any more (for good reasons). These days, only a few things can be set on the CSS :visited selector, mostly colour changes. Changing and setting colours in general is dangerous, especially in a world where so many websites already set them to various unpredictable values that may not be distinct from my new ones.

Given the limitations on what you can set on :visited, I most likely need both an overall distinctive style for a:link and then a complementary modification for :visited. While MDN's Styling links has given me some things to think about, I currently haven't come up with a good styling combination that actually works.

(In the process of poking around MDN, I did discover that in Firefox you can now access the standard colours of visited and unvisited links with the special colour names of LinkText and VisitedText.)