Rust 1.x seems to not always be backward compatible in practice

June 13, 2021

For a long time now, I've needed (or at least wanted) to simultaneously put a stress load on a test NFS fileserver and a stress CPU and memory load on another machine. Back in the old days on Linux, I would have done this by compiling the Linux kernel repeatedly, but in today's modern age the Linux kernel isn't big enough any more; it builds too fast and isn't big enough. I eventually settled on repeatedly building Firefox from source, partly because it's something I'm already familiar with. Specifically, on one of our Ubuntu 18.04 LTS machines I wound up building the then current release version of Firefox with a then current Rust version installed into a custom location with Rustup. Everything worked fine for a long time, and then one day I made the mistake of absently deciding to run 'rustup upgrade' on my custom Rust location. My Firefox builds immediately blew up with a series of Cargo errors and then later Rust errors.

I was unable to fix this and restore my Firefox build environment to a working state. In particular, it appears that older versions of Firefox (beyond some point I didn't try to determine precisely) can't be built with modern versions of Rust and Cargo (which is tightly coupled to Rust). You can to some extent patch Cargo.toml files so that Cargo will accept them, but the Rust errors were beyond my ability to deal with. My eventual solution was to obtain the current Firefox release, the current Rust release, and run Firefox's own 'build a custom virtual environment with all of your build dependencies' script. Someday this too will break, and in the mean time any performance measurements from before this transition aren't compatible with those after it, since I'm building different things with a different toolchain.

All of this was very startling to me, especially the Rust compile time errors. I'm used to Go, where there is a strong commitment to keeping old Go code building properly even as Go moves forward (but unfortunately less commitment to how the Go equivalent of Cargo works). Rust is evidently moving more aggressively than that to develop their language and are willing to invalidate older code if they feel strongly enough about this. When I am in a bad mood, I feel that means that 'Rust 1.x' is not really '1.x' as I would consider it. Either Rust has not really reached 1.0 yet or there have been a whole series of '2.0', '3.0' and so on Rust releases that are not actually labeled as such.

Update: Much of this is a Firefox problem, where Mozilla has apparently specifically set an environment variable to enabled unstable and changing Rust features in the stable Rust compiler. The effect is to tie specific Firefox versions to a narrow range of Rust stable compiler versions that implement the specific unstable features that Firefox's code expects.

I'm sure that Rust people have a whole series of good explanations for why all of this was necessary and why it's a good thing that I can no longer build older versions of Firefox (for instance, some of the Rust errors may be for things that were unsafe but not previously detected). All I can say is that as an outsider, the resulting experience is not a particularly good one.

(Rust is not the only language and build environment that can suffer from this; C compilers can give you issues if you tell them to abort on warnings.)

(This elaborates on some Tweets of mine at the time.)

Comments on this page:

By nyanpasu64 at 2021-06-13 12:38:48:

Do you have a screenshot or text transcript of the exact cargo and rustc errors?

By cks at 2021-06-13 13:24:24:

I went through this exercise two months ago and didn't take or retain notes from it. However, it should be easy to reproduce; get an older Firefox, say Firefox 65 (from January 2019) or maybe even Firefox 78 (the current ESR), and (try to) build them with a current Rust. I believe the first obstacle will be a Cargo.toml issue and then if you patch or adopt it, you get into the Rust errors.

(I've heard that Arch has some significant patches for their version of Firefox 78 ESR. However, I've also heard that Firefox apparently deliberately enables unstable and changing features in the Rust compiler and so this is actually not (stable) Rust's fault.)

By cks at 2021-06-13 18:51:13:

The Cargo error for Firefox 78 building on 20.04 is:

0:53.35 b'ERROR: Couldn\'t execute `cargo metadata` with manifest "/tmp/mozilla-release/toolkit/library/rust/Cargo.toml": Metadata(Output { status: ExitStatus(ExitStatus(25856)), stdout: "", stderr: "error: failed to parse manifest at `/tmp/mozilla-release/Cargo.toml`\\n\\nCaused by:\\n dependency (nix) specification is ambiguous. Only one of `branch`, `tag` or `rev` is allowed.\\n" })\nERROR: Couldn\'t generate bindings for /tmp/mozilla-release/toolkit/library/rust.\n'

(I went to Ubuntu 20.04 because 18.04 has an ancient version of nodejs that even fairly old versions of Firefox don't like.)

Then Rust 1.52.1 errors started with:

Compiling style_traits v0.0.1 (servo/components/style_traits)
error: expected literal
  --> servo/components/style_traits/
12 | / define_css_keyword_enum! {
13 | |     pub enum UserZoom {
14 | |         Zoom = "zoom",
15 | |         Fixed = "fixed",
16 | |     }
17 | | }
   | |_^
   = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
By nyanpasu64 at 2021-06-13 23:16:12:

Ouch, I don't know what those errors are (since it seems to be a macro rather than regular syntax, and getting a macro backtrace requires nightly Rust which sucks). For what it's worth, the define_css_keyword_enum! macro_rules was changed (not removed) in And the macro_rules! syntax isn't unstable, so a compatibility break would be unfortunate if true.

By 1vader at 2021-06-14 01:10:19:

As far as I can tell, this is not at all an issue related to Rust. This all seems to be caused by Firefox's liberal use of unstable features and using wonky setups to allow using them even on stable compilers. See the Reddit discussion for some more details:

There have been some places where backwards compatibility has been broken but as far as I'm aware it was always either because of a bug or security issue or had something to do with a specific feature or very particular setup or situation that only occured very rarely or never in practice. Rust uses crater to build all (!) packages published on when problematic changes are introduced (and also just for every release in general) and checks for any regressions in the result. In some cases, the team will simply contribute fixes to the few affected crates which I guess would still break building old versions of that package with a new compiler but that's very rare in practice. Just look at all the hoops the team went through to ensure the iterator implementation on arrays wouldn't break anything, even though the current behavior has always been stupid in the first place with a simple and consistent alternative and had a deprecation warning for quite a while.

Written on 13 June 2021.
« How we're dealing with our expiring OpenVPN TLS root certificate
A strong commitment to backwards compatibility means keeping your mistakes »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jun 13 00:04:46 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.