Wandering Thoughts archives

2018-01-28

Adding 'view page in no style' to the WebExtensions API in Firefox Quantum

After adding a 'view page in no style' toggle to the Firefox context menu, I still wasn't really satisfied. Having it in the context menu is better than nothing, but I'm very used to using a gesture for it and I really wanted that in Firefox Quantum (even if I'm not using Firefox 57+ today, I'm going to have to upgrade someday). My first hope was that the WebExtensions API exposed some way to call Firefox's sendAsyncMessage(), so I skimmed through the Firefox WebExtensions API documentation. Unsurprisingly, Firefox does not expose such a powerful internal API to WebExtensions, but I did find browser.tabs.toggleReaderMode(). At this point I had a dangerous thought: how difficult would it be to add a new WebExtensions API to Firefox?

It turns out that it's not that difficult and here is how I did it, building on the base of my context menu hack. Since I already have the core code to toggle 'view page in no style' on and off, all I need is a new API function. The obvious place to start is with how toggleReaderMode is specified and implemented, because it should be at least very close to what I need.

Unfortunately there are a lot of mentions of 'toggleReaderMode' in various files, so we need to look around and guess a bit:

; cd mozilla-central
; rg toggleReaderMode
[...]
browser/components/extensions/schemas/tabs.json
991:        "name": "toggleReaderMode",
[...]

The WebExtensions API has to be defined somewhere and looking at tabs.json pretty much confirms that this is it, as you can see from the full listing. The relevant JSON definition starts out:

{
  "name": "toggleReaderMode",
  "type": "function",
  "description": "Toggles reader mode for the document in the tab.",
  "async": true,
  "parameters": [
  [...]

It's notable that this doesn't seem to define any function name to be called when this API point is invoked, which suggests that the function just has the same name. So we can search for some function of that name:

; rg toggleReaderMode
[...]
browser/components/extensions/ext-tabs.js
1045:        async toggleReaderMode(tabId) {
[...]

This is very likely to be what we want, given the file path, and indeed it is. Looking at the full source to toggleReaderMode() shows that it works roughly how I would expect, in that it winds up calling sendAsyncMessage("Reader:ToggleReaderMode"). This means that all we need to turn it into our new toggleNoStyleMode() API is the trivial modification of changing the async message sent, and of course the name.

First, the implementation, more or less blindly copied from toggleReaderMode:

// <cks>: toggle View Page in No Style
async toggleNoStyleMode(tabId) {
  let tab = await promiseTabWhenReady(tabId);
  tab = getTabOrActive(tabId);

  tab.linkedBrowser.messageManager.sendAsyncMessage("PageStyle:Toggle");
},

toggleReaderMode() has code to deal with the possibility that the tab can't be put in Reader mode, which we've taken out; as before, we toggle things unconditionally without caring if it's applicable. This is probably not what you should do for a proper API, but this is a hack.

Having implemented the API function, we now need to make it part of the actual WebExtensions API by changing tabs.json. We use a straight copy of toggleReaderMode's API specification with the name and description changed (the latter just for neatness):

{
  "name": "toggleNoStyleMode",
  "type": "function",
  "description": "Toggles view page in no style mode for the document in the tab.",
  "async": true,
  "parameters": [
    {
      "type": "integer",
      "name": "tabId",
      "minimum": 0,
      "optional": true,
      "description": "Defaults to the active tab of the $(topic:current-window)[current window]."
    }
  ]
},

With my Firefox Quantum rebuilt with these changes included, I could now add a user script to Foxy Gestures to test and use this. Following the examples of common user scripts, especially the 'Go to URL' example, the necessary JavaScript code is:

 executeInBackground(() => {
    getActiveTab(tab => browser.tabs.toggleNoStyleMode(tab.id));
 });

(Since the tabId argument is optional and probably defaults to what I want, I could probably simplify this down to just calling browser.tabs.ToggleNoStyleMode() with no argument and without the getActiveTab() dance. But I'm writing all of this mostly by superstition, so carefully copying an existing working example doesn't hurt.)

Actually doing all of this and testing it immediately showed me a significant limitation of 'view page in no style' in Firefox Quantum, which is that when Firefox disables CSS for a page this way, it really disables all CSS. Including and especially the CSS that Foxy Gestures has to inject in order to show mouse trails for mouse gestures that you're in the process of making. The gestures still work, but I have to make them blindly. This is probably okay for my usage, but it's an unfortunate limitation in general. Perhaps Mozilla would accept a bug report that View → Page Style → No Style also affects addons.

(If I want another option, uMatrix allows you to disable CSS temporarily, but it takes a lot more work. And in Quantum it might still affect addon-injected CSS; I haven't experimented.)

(This and previous entries extend, at great length, my tweets. They definitely took more time to write than it took me to actually do both hacks.)

PS: I don't know how WebExtensions permissions are set up and controlled, but apparently however it works, Foxy Gestures didn't need any new permissions to access my new API. The MDN page on WebExtensions permissions suggests that because I put my new API in browser.tabs, it's available without any special permissions.

FirefoxNewWebExtsAPI written at 22:50:42; Add Comment

Adding 'view page in no style' to Firefox Quantum's context menu

Yesterday I covered the limitations of Firefox's Reader mode, why I like 'view page in no style' enough to make it a personal FireGestures user script gesture, and how that's impossible in Firefox 57+ because WebExtensions doesn't expose an API for it to addons. If I couldn't have this accessible through Foxy Gestures, I decided that I cared enough to hack my personal Firefox build to add something to toggle 'view page in no style' to the popup context menu (so that it'd be accessible purely through the mouse in my setup). Today I'm going to write up more or less how I did it, as a guide to anyone else who wants to do such modifications.

An important but not widely known thing about Firefox that makes this much more feasible than it seems is that a great deal of Firefox's UI (and a certain amount of its actual functionality) is implemented in JavaScript and various readable configuration files (XUL and otherwise), not in C++ code. This makes it much easier to modify and add to these things relatively blindly, and as a bonus you're far less likely to crash Firefox or otherwise cause problems if you get something wrong. I probably wouldn't have attempted this hack if it had involved writing or modifying C++ code, but as it was I was pretty confident I could do it all in JavaScript.

First we need to find out how and where Firefox handles viewing pages in no style. When I'm spelunking in the Firefox code base, my approach is to start from a string that I know is used by the functionality (for example in a menu entry that involves whatever it is that I want) and work backward. Here the obvious string to look for is 'No Style'.

; cd mozilla-central
; rg 'No Style'
[...]
browser/locales/en-US/chrome/browser/browser.dtd
708:<!ENTITY pageStyleNoStyle.label "No Style">

(rg is ripgrep, which has become my go-to tool for this kind of thing. You could use any recursive grep command.)

Firefox tends not to directly use strings in the UI; instead it adds a layer of indirection in order to make translation easier. Looking at browser.dtd shows that it just defines some text setup stuff, with no actual code, so we want to find where pageStyleNoStyle is used. Some more searching finds browser/base/content/browser-menubar.inc, where we can find the following fairly readable configuration text:

<menupopup onpopupshowing="gPageStyleMenu.fillPopup(this);">
  <menuitem id="menu_pageStyleNoStyle"
            label="&pageStyleNoStyle.label;"
            accesskey="&pageStyleNoStyle.accesskey;"
            oncommand="gPageStyleMenu.disableStyle();"
            type="radio"/>
[...]

The important bit here is oncommand, which is the actual JavaScript that gets run to disable the page style. Unfortunately this just runs a function, so we need to search the codebase again to find the function, which is in browser/base/content/browser.js:

 disableStyle() {
   let mm = gBrowser.selectedBrowser.messageManager;
   mm.sendAsyncMessage("PageStyle:Disable");
 },

Well, that's not too helpful, since all it does is send a message that's handled by something else. We need to find where this message is handled, which we can do by searching for 'PageStyle:Disable'. That turns up browser/base/content/tab-content.js, where we find:

var PageStyleHandler = {
 init() {
   addMessageListener("PageStyle:Switch", this);
   addMessageListener("PageStyle:Disable", this);
   [...]

 receiveMessage(msg) {
   switch (msg.name) {
   [...]
     case "PageStyle:Disable":
       this.markupDocumentViewer.authorStyleDisabled = true;
       break;
   [...]

Since I wanted my new context menu entry to toggle this setting, I decided that the simple way was to add a new message for this. That needs one line in init() to register the message:

   addMessageListener("PageStyle:Toggle", this); // <cks>

and then a new case in receiveMessage() to handle it:

     // <cks>
     case "PageStyle:Toggle":
       this.markupDocumentViewer.authorStyleDisabled = !this.markupDocumentViewer.authorStyleDisabled;
       break;

(I tend to annotate my own additions and modifications to Firefox's code so that I can later clearly identify that they're my work. Possibly this case should have a longer comment so that I can later remember why it's there and what might make it unneeded in the future.)

We can definitely disable the page style by setting authorStyleDisabled to true, but it's partly a guess that we can re-enable the current style by resetting authorStyleDisabled to false. However, it's a well informed guess since Firefox 56 and before worked this way (which I know from my FireGestures user script). It's worth trying, though, because duplicating what PageStyle:Switch does would be much more complicated.

Next I need to hook this new functionality up to the context menu, which means that I have to find the context menu. Once again I'll start from some text that I know appears there:

; rg 'Save Page As'
[...]
browser/locales/en-US/chrome/browser/browser.dtd
567:<!ENTITY savePageCmd.label            "Save Page As…">

There's a number of uses of savePageCmd in the Firefox source code, because there's a number of places where you can save the page, but the one I want is in browser/base/content/browser-context.inc (which we can basically guess from the file's name, if nothing else). Here's the full menu item where it's used:

<menuitem id="context-savepage"
          label="&savePageCmd.label;"
          accesskey="&savePageCmd.accesskey2;"
          oncommand="gContextMenu.savePageAs();"/>

At this point I had a choice in how I wanted to implement my new context menu item. As we can see from inspecting the oncommand for this entry (and others), the proper way is to add a new toggleNoStyle() function to gContextMenu that sends our new PageStyle:Toggle message. The hack way is to simply write the necessary JavaScript inline in the oncommand for our new menu entry. Let's do this the proper way, which means we need to find gContextMenu and friends.

Searching for savePageAs and hidePlugin (from another context menu entry) says that they're defined in browser/base/content/nsContextMenu.js. So I added, right after hidePlugin(),

 // <cks> Toggle view page in no style
 toggleNoStyle() {
   let mm = gBrowser.selectedBrowser.messageManager;
   mm.sendAsyncMessage("PageStyle:Toggle");
 },

(This is simply disableStyle() modified to send a different asynchronous message. Using gBrowser here may or may not be entirely proper, but this is a hack and it seems to work. Looking more deeply at other code in nsContextMenu.js suggests that perhaps I should be using this.browser.messageManager instead, and indeed that works just as well as using gBrowser. I'm preserving my improper code here so you can see my mis-steps as well as the successes.)

Now I can finally add in a new context menu entry to invoke this new gContextMenu function. Since this is just a hack, I'm not going to define a new DTD entity for it so it can be translated; I'm just going to stick the raw string in, and it's not going to have an access key. I'm putting it just before 'Save Page As' for convenience and so I don't have to worry that it's in a section of the context menu that might get disabled on me. The new menu item is thus just:

<menuitem id="context-pagestyletoggle"
          label="Toggle Page Style"
          oncommand="gContextMenu.toggleNoStyle();"/>

(I deliberately made the menu string short so that it wouldn't force the context menu to be wider than it already is. Probably I could make it slightly more verbose without problems.)

After rebuilding my local Firefox Quantum tree and running it, I could test that I had a new context menu item and that it did in fact work. I even opened up the browser console to verify that my various bits of JavaScript code didn't seem to be reporting any errors.

(There were lots of warnings and errors from other code, but that's not my problem.)

This is a hack (for reasons beyond a hard-coded string). I've made no attempt to see if the current page has a style that we can or should disable; the menu entry is unconditionally available and it's up to me to use it responsibly and deal with any errors that come up. It's also arguably in the wrong position in the context menu; it should probably really go at the bottom. I just want it more accessible than that, so I don't have to move my mouse as far down.

(Not putting it at the bottom also means that I don't need to worry about how to properly interact with addons that also add context menu entries. Probably there is some easy markup for that in browser-context.inc, but I'm lazy.)

PS: My first implementation of this used direct JavaScript in my oncommand. I changed it to the more proper approach for petty reasons, namely that putting it in this entry would have resulted in an annoyingly too-wide line of code.

FirefoxNoStyleInContext written at 20:28:55; Add Comment

2018-01-18

Why Go cares about the difference between unsafe.Pointer and uintptr

Go has two things that are more or less the representation of an untyped pointer; uintptr and unsafe.Pointer (which, contrary to appearances, is a built-in type). On the surface this is a little bit odd, because unsafe.Pointer and uintptr can be converted back and forth between each other. Why not have only one representation of a pointer? What's the difference between the two?

The superficial difference is that you can do arithmetic on an uintptr but not on an unsafe.Pointer (or any other Go pointer). The important difference is explicitly pointed out by the unsafe package's documentation:

A uintptr is an integer, not a reference. Converting a Pointer to a uintptr creates an integer value with no pointer semantics. Even if a uintptr holds the address of some object, the garbage collector will not update that uintptr's value if the object moves, nor will that uintptr keep the object from being reclaimed.

Although unsafe.Pointers are generic pointers, the Go garbage collector knows that they point to Go objects; in other words, they are real Go pointers. Through internal magic, the garbage collector can and will use them to keep live objects from being reclaimed and to discover further live objects (if the unsafe.Pointer points to an object that has pointers of its own). Due to this, a lot of the restrictions on what you can legally do with unsafe.Pointers boil down to 'at all times, they must point to real Go objects'. If you create an unsafe.Pointer that doesn't, even for a brief period of time, the Go garbage collector may choose that moment to look at it and then crash because it found an invalid Go pointer.

By contrast, a uintptr is merely a number. None of this special garbage collection magic applies to objects 'referred to' by a uintptr because as just a number, a uintptr doesn't refer to anything. In turn this leads to many of the careful limitations on the various ways that you can turn an unsafe.Pointer into a uintptr, manipulate it, and turn it back. The basic requirement is that you do this manipulation in such a way that the compiler and the runtime can shield the temporary non-pointerness of your unsafe.Pointer from the garbage collector so this temporary conversion will be atomic with respect to garbage collection.

(I think that my use of unsafe.Pointer in my entry on copying blobs of memory into Go structures is safe, but I admit I'm now not completely sure. I believe that there is some magic going on with cgo, since it can safely manufacture unsafe.Pointers that point to C memory, not Go memory.)

PS: As of Go 1.8, all Go pointers must always be valid (I believe including unsafe.Pointers), even if a garbage collection isn't running at the time. If you ever have an invalid pointer stored in a variable or a field, your code can crash merely by updating the field to a perfectly valid value, including nil. See for example this educational Go bug report.

(I was going to try to say something about the internal magic that allows the garbage collector to cope with untyped unsafe.Pointer pointers, but I'm convinced I don't understand enough about it to even say what sort of magic it uses.)

GoUintptrVsUnsafePointer written at 20:19:27; Add Comment

2018-01-02

Some notes on relative imports and vendor/ in Go

For reasons beyond the scope of this entry, I've become interested in ways to work with multiple packages in Go without doing it in the normal way, with a $GOPATH/src and packages and programs that can be go get'd from Github or any of the other places that the go tool supports. The short version of what I'm interested in is that I'd like to create self contained program source code trees that are still modularized into multiple Go packages, instead of throwing everything into 'package main'.

(If you put everything in main, you can put all your source code in one directory and then tell people to just cd there and do 'go build' to build it.)

Go has two plausible mechanisms for this, vendoring and the rarely used relative import path (which is more properly a filesystem path). I'll start with the latter. Relative import paths come from 'go help packages':

An import path that is a rooted path or that begins with a . or .. element is interpreted as a file system path and denotes the package in that directory.

(Italics mine.)

Suppose that you have a program and you want to have two sub-packages, a and b. Then you can put together a directory tree like this:

program/
   main.go
   a/a.go
   b/b.go

Your main.go will use 'import "./a" to make a relative import from the current directory, while your a.go would use 'import "../b" to reach package b (if it needed to). You have to put a.go and b.go into appropriate subdirectories, even if each package only has a single file, because even this sort of import applies to the entire directory.

However, relative imports turn out to have one drawback, and that's that you can't use them in a package or program that has its source code under your $GOPATH. If you do, you'll get an error message:

main.go:3:8: local import "./a" in non-local package

(Note that you get this error even if you are in the directory itself and just running 'go build', to build the current directory. It's possible that this is new behavior in Go 1.10, but if so, well, you're going to have to use Go 1.10 or later at some point.)

The other plausible mechanism is vendoring. Given the same main program and two sub-packages a and b, construct your directory tree as:

program/
   main.go
   vendor/
      a/a.go
      b/b.go

Now everything simply does 'import "a"' or 'import "b", which works in both main.go and a.go.

The drawback of vendoring is the exact reverse of relative imports; instead of not working under your $GOPATH, it only works under your $GOPATH. This unfortunately makes vendoring not entirely useful for my particular purpose here, because if I'm going to require a $GOPATH, I might as well use something approximating a proper Go package hierarchy:

src/cslab/program
             cmd/program/main.go
             a/a.go
             b/b.go

(The whole cmd/program subdirectory tree feels a bit excessive here, but I'm not sure if there's a generally accepted better way.)

With this setup I can refer to things as 'import "cslab/program/a"' and so on. There's relatively little need to hide them in a vendor/ subdirectory just so I can confuse everyone with 'import "a"'.

In fact, I think this potential confusion from vendoring makes it better to use relative imports if you have to pick one of these two options, because at least when people see them it's fairly clear what they mean.

(It would be useful if the vendor/ subdirectory worked even outside $GOPATH, but it doesn't currently so even if this was accepted as a bug by the Go people it would be some time before a fix appeared and was more or less universally usable.)

GoOddImportsNotes written at 03:57:49; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.