2018-05-21
Bad versions of packages in the context of minimal version selection
Recently, both Sam Boyer and Matt Farina have made the point that Go's proposed package versioning lacks an explicit way for packages to declare known version incompatibilities with other packages. Suppose that you have a package A and it uses package X, initially at v1.5.0. The package X people release v1.6.0, which is fine, and then v1.7.0, where they introduce an API behavior change that is incompatible with how your package uses the API (Matt Farina's post has a real world example of this). By the strict rules of semantic versioning this is a no-no, but in real life it happens for all sorts of reasons. People would like the ability to have their own package say 'I'm not compatible with v1.7.0 (and later versions)', which Russ Cox's proposal doesn't provide.
The first thing to note is that in a minimal version selection environment, this incompatibility doesn't even come up as long as you're only building the package or something using the package that has no other direct or indirect dependencies on package X. If you're only using package A, package A says it wants X@v1.5.0 and that's what MVS picks. MVS will never advance to the incompatible version v1.7.0 on its own; it must be forced to do so. Even if you're also using package B and B requires X@v1.6.0, you're still okay; MVS will advance the version of X but only to v1.6.0, the new minimal version.
(This willingness to advance the version at all is a pragmatic tradeoff. We can't be absolutely sure that v1.6.0 is really API compatible with A's required X@v1.5.0, but requiring everyone to use exactly the same version of a package is a non-starter in practice. In order to make MVS useful at all, we have to hope that advancing the version here is safe enough (by default, and if we lack other information).)
So this problem with incompatible package versions only comes up in MVS if you also have another package B that explicitly requires X@v1.7.0. The important thing here is that this version incompatibility is not a solvable situation. We cannot build a system that works; package A doesn't work with v1.7.0 while package B only works with v1.7.0 and we need both. The only question is whether MVS or an MVS-like algorithm will actually tell us about this problem, aborting the build, or whether it will build a system that doesn't work (if we're lucky the system will fail our tests).
To me, this changes how critical the problem is to address. Failure to build a working system where it's possible would be one thing, but we don't have that; instead we merely have the question of whether you're going to get told up front that what you want isn't possible.
The corollary to this is that when package A publishes information that it's incompatible with X version v1.7.0, it's doing so almost entirely as a service for other people, not something it needs for itself. Since A's manifest only requires X@v1.5.0, MVS will generally use v1.5.0 when building A alone (let's assume that none of A's other dependencies also use X and will someday advance to requiring X@v1.7.0). It's only when A gets bundled together with B that problems happen, and so this is mostly when A's information about version incompatibility is useful. Should this information be published in a machine readable form? Well, I think it would be nice, but it depends on what else we have to give up for it.
(The developers of A may want to leave themselves a note about the situation in their version manifest, of course, just so that no developer accidentally tries advancing X's version and then gets surprised by the results.)
PS: There is also an argument that such incompatible version blocks should only be advisory warnings or the like. As the person building the overall system, you may actually know that the end result will work anyway; perhaps you've taken steps to compensate for the API incompatibility in your own code. Since the failure is an overall system failure, package A can't necessarily be absolutely sure about things.
(Things might be easier to implement as advisory warnings. One approach would be to generate the MVS versions as usual, then check to see if anyone declared an incompatibility with the concrete versions chosen. Resolving the situation, if it's even possible, would be up to you.)