Wandering Thoughts archives

2022-09-28

How I've set up my libvirt based virtual machines (in late 2022)

I moved from VMWare Workstation to using Linux's libvirt and its native virtualization facilities earlier this year, and I've been happy with that move although I still would like to be able to take good snapshots of UEFI based virtual machines. Over time I've wound up with a setup that I'm happy with for the work that I do, one that's similar but not quite the same as my VMWare setup.

I have two groups of VMs. One group is Fedora VMs (and one Windows VM) that I use for testing Fedora upgrades, Windows things, and so on. All of these machines are NAT'd through a somewhat customized NAT setup, and I basically just run them. I make little to no usage of VM snapshots; about my only use is to snapshot them before I do something I consider unusually risky (or that I may want to re-try), and then generally to delete the snapshot later after it worked.

The second group of VMs is VMs used to test various things in our Ubuntu server environment. Our environment expects machines to have real IPs, so all of these vms use 'macvtap' bridged networked (on a second, completely unused port) and have their own IPs. Our standard Ubuntu install setup has a two stage install process, where we first install from ISO image (which sets the machine's IP address, among other things) and then run through a large postinstall step to customize machines. With most of the testing I do, I want to start from scratch in a fresh install (which most closely mimics real servers) rather than try to shuffle around software and setups on an already installed machine.

To make this more convenient, I've made snapshots for each machine with one or more Ubuntu versions installed from our ISO image (with the machine's IP address set) but no further post-install setup done. At the very start of a VM's life, I've also made an initial snapshot of it with an empty disk. These snapshots are given imaginative yet obvious names like 'empty-initial' and '2204-initial', and when I want to use a vm for something based on 22.04 or 20.04 or whatever, I pick one that's not currently in use and do 'virsh snapshot-reset <what> 2204-initial; virsh start <what>' and proceed from there. Sometimes, if I'm going to be working with a particular setup of a machine I will also create a snapshot for it in that state and then potentially keep returning to it, but that's more chancy because of configuration and package version drift.

(VMWare Workstation visualized the equivalent snapshots as a tree. I don't know if the underlying QEMU disk snapshots or the libvirt versions have any sort of tree to them, but I don't miss the tree view; it's generally straightforward enough to keep track of them from their names.)

I liked libvirt well enough to set up a general virtualization server for tests, scratch machines, and so on, which also used macvtap bridged networking on a host port that's dedicated to this. The virtual machines on this server follow the same general pattern, although this time I set up enough VMs that each VM can be more or less dedicated to a specific Ubuntu version instead of switching back and forth between different ones. At the moment, some of these VMs have specific ongoing testing purposes, but in general what's on them is supposed to be completely expendable. In their role for ongoing testing, some of these machines are normally running all the time. On my desktop I don't normally leave VMs running, although sometimes I lose track of that and don't notice one for a while.

(I should make snapshots of the current state of these ongoing test VMs just in case.)

Although libvirt can (I believe) snapshot a live and running machine, I always make snapshots with the machine off. Partly this is habit from my days of using VMWare Workstation, but partly it's because our server setups don't expect to be (effectively) suspended and then woken again some time later. It seems more reliable to shut them down then have them boot up (much) later, which are things we expect to have happen and so design our scripts and so on for.

(Although PXE based network installs might simplify some of this, we don't use them and they would still take more time than starting from a partially prepared image.)

linux/LibvirtMySetup2022 written at 21:52:16; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.