Wandering Thoughts archives

Sort of making snapshots of UEFI libvirt-based virtual machines

As of early 2022, one of the limitations of libvirt is that it doesn't support making snapshots of virtual machines that use UEFI instead of BIOS booting. Since various people want to get rid of BIOS booting, this is a problem on several levels. Fortunately it's possible to sort of get around this, in one of two ways; the difficult and I believe incomplete way that I haven't gotten working, and the easier way that I have. All of this assumes that you're using the normal QEMU/KVM backend for libvirt (which supports UEFI via OVMF).

The difficult way is to shut down your UEFI VM and use virsh to make a 'disk-only snapshot' of your UEFI-based VM. Making the snapshot is quite straightforward:

virsh snapshot-create-as <domain> --name <snap-name> --disk-only

The difficult problem comes when you want to revert to a disk only snapshot that's been made this way. By default, disk-only snapshots are what is called 'external snapshots', where the snapshot is implemented by creating a new disk image file that's layered on top of the original and then modifying the domain's metadata to use it (see this libvirt wiki page). To revert to the original you must change the domain's metadata by hand; one discussion of this is here (which has extensive links). Because of the mechanics of this, I suspect that doing such a reversion invalidates any other snapshots you have.

The easier way is to shut down your UEFI VM and then use qemu-img to manually make QEMU 'internal snapshots' of your disks, following the directions of (for example) this qemu-img cheatsheet. Internal snapshots are entirely contained within the QEMU qcow2 disk image, so libvirt is basically oblivious to them; it will have QEMU run your VM using the disk image with the disk image in whatever state qemu-img put it in. Since QEMU itself knows what to do with internal snapshots, everything works out.

However, this only gives you disk snapshots (and only of shut down VMs), and everything about managing them is up to you. Neither virt-manager nor virsh will give you any hint that you have QEMU-level disk snapshots, and you will have to keep track of them and manipulate them directly with qemu-img commands (which will probably require you to be root even if you've arranged for virsh to work with you as a regular user). For my own future use, the very short cheat sheet for listing, creating, reverting to, and destroying these snapshots is:

qemu-img snapshot -l <file>.qcow2

qemu-img snapshot -c <snap-name> <file>.qcow2

# -a means 'apply' (revert to)
qemu-img snapshot -a <snap-name> <file>.qcow2

qemu-img snapshot -d <snap-name> <file>.qcow2

(As I write this, the current 'qemu-img snapshot' documentation says very little more than this.)

You can use qemu-img to revert to a snapshot without destroying snapshots based on top of it; you just 'apply' the snapshot and it all works. I haven't investigated the disk space usage consequences of this, but I assume it's no worse than the regular libvirt snapshots I'm using on BIOS based VMs because those are also internal snapshots.

(Libvirt can list snapshots in a tree, with 'virsh snapshot-list --tree <vm-name>', but I don't see a way to make qemu-img do this and so I suspect the idea of a tree may be entirely inside libvirt.)

In general this is a relatively long standing RFE in Fedora, bug #1881850. That bug may someday report that a libvirt feature to support this has landed, especially in light of Fedora proposing to deprecate BIOS booting. See also RHEL bug #1519002 on improving libvirt support for external snapshots.

My own personal way of dealing with this is that I don't plan to use UEFI based VMs unless I absolutely have to test something UEFI specific. My VMs will be booting via BIOS for as long as that's supported by Ubuntu and Fedora.

(This is the extended discussion version of some tweets. Partly because I want to write down both the logic and the commands somewhere where I'm sure I can find them again, since I'm sure I'll be forced to do more with UEFI VMs over time.)

PS: GNOME Boxes has been annoyed enough about this issue (and the serious problems with the user experience it createst) to disable support for UEFI VMs.

Sidebar: virsh internal disk-only snapshots don't work for this

Virsh can explicitly make disk-only internal snapshots of your disks, for example (this command line is untested):

virsh snapshot-create-as <vm-name> <snap-name> --disk-only --diskspec vda,snapshot=internal

However, it will refuse to make internal disk-only snapshots of UEFI machines, reporting an error like this one:

Error creating snapshot: Operation not supported: Internal snapshots of a VM with pflash based firmware are not supported.

This is clearly a libvirt limitation since qemu-img doesn't care. But so it goes.