The good and bad of IPS (as I see it)

June 11, 2013

IPS (the 'Image Packaging System') is the new packaging system used in Solaris 11 and (more importantly) many Illumos-derived distributions; it replaces Solaris 10 packages and patches. I have previously described IPS as being more or less like git; it puts all files together in a hash-based content store and then has 'packages' that are basically just indexes into the store. This contrasts with the traditional Linux approach to packaging where each package is an archive of some sort that contains all actual files in the package.

The attractive part of IPS is what the content store approach does for repositories and for package updates. If files are the same between two versions of a package (or between multiple packages), the repository only needs to store one copy and the package update or install process can detect that you already have the needed file installed already. This mimics the practical behavior of Solaris 10 patches, which only included changed files (as opposed to the Linux approach, where changing just one file in a package causes you to re-issue an entire second copy of the whole package).

(This also minimizes what needs to be digitally signed. Much as in git, you don't need to digitally sign the files themselves, just the package index data. The all-in-one Linux package format means that you generally need to sign and verify large blobs of data.)

The bad part of IPS is what it does to downloading and storing packages. As far as I know, files are downloaded from IPS repositories in the same way that they're stored; you ask for them one by one and they then dribble in bit by bit. As we've learned the hard way, this is not a great way to do things on the modern Internet (or in general) because each separate fetch requires a new connection (or at least a new request) and that has various consequences.

(IPS packages are normally fetched over HTTP or HTTPS but I don't know if the IPS client and server are smart enough to take advantage of HTTP connection reuse.)

I'm also not enthused about how this makes package repositories harder to manage and exposes them to subtle forms of breakage (such as a file that's listed in package manifests but not present in the repository). Pruning old packages is now necessarily a whole-repository operation, since you can't just remove their files without seeing if any other package uses them.

I suspect that Sun developed IPS this way to preserve the small sizes and small installation changes of Solaris 10 patches (which transfer and install only the changed files instead of the whole package). I prefer the simpler approach of Linux packages (and I note that Linux package updates themselves can optimize both transfer size and install time actions).


Comments on this page:

From 89.70.184.230 at 2013-06-11 03:07:22:

The all-in-one Linux package format means that you generally need to sign and verify large blobs of data.

Not quite. APT only needs Release file to be signed. Release file points to Packages files for different architectures (and sources), and this in turn points to individual packages. All pointing is done through hashes, so you get pretty much the same as with signing git commit.

Of course individual packages can be signed digitally, but APT with signatures doesn't require this.

By cks at 2013-06-11 15:38:27:

It's worth noting that it's very important to sign packaging metadata (and to date it securely as well, so that clients can tell if they are getting fed old metadata).

From 86.161.172.220 at 2013-06-14 07:29:30:

- you can create a file which contains a single package with all its files and then install directly from it

- the archive file can actually include multiple packages so for example you can have a single file with all dependencies required by a given software, or you can split them to single files like usually is the case on Linux; for example Symantec delivers VCS as a single IPS archive files with all required packages and files contained in it. You can selectively install only what you need if that's what you want

- downloading packages - it actually opens multiple connections in parallel and I think it does also re-use connections, underneath it uses libcurl which does support HTTP reuse

- you can also install/download packages directly from a file system, for example over nfs

- deleting packages from a repository - just use pkgrepo which does the job for you and it just works even on very large repositiories

see also:

http://timsfoster.wordpress.com/2012/10/04/ips-changes-in-solaris-11-1-brought-to-you-by-the-letter-p/

http://timsfoster.files.wordpress.com/2010/10/ips-solaris-day_nz.pdf

Written on 11 June 2013.
« What the SELinux community should be doing
UI details that you don't expect to can matter a lot »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Jun 11 00:11:54 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.