Some notes on moving a software RAID-1 root filesystem around (to SSDs)
A while ago I got some SSDs for my kind of old home machine but didn't put them to immediate use for various reasons. Spurred on first by the feeling that I should get around to it sometime, before my delay got too embarrassing, and then by one of my system drives apparently going into slow IO mode for a while, I've now switched my root filesystem over to my new SSDs. I've done this process before, but this time around I want to write down notes for my future reference rather than having to re-derive all the steps each time. All of this is primarily for Fedora, currently Fedora 25; some steps will differ on other distributions such as Ubuntu.
I partitioned using GPT partitions, not particularly because I needed to with 750 GB SSDs but because it seemed like a good idea. I broadly copied the partitioning I have on my SSDs at work for no particularly strong reason, which means that I set it up this way:
Number Size Code Name 1 256 MB EF00 EFI System 2 1 MB EF02 BIOS boot partition 3 100 GB FD00 Linux RAID 4 1 GB FD00 Linux RAID (swap) 5 <rest> BF01 ZFS
Some of this is likely superstition by now, such as the BIOS boot partition.
With the pair of SSDs partitioned, I set up the software RAID-1
arrays for the new /
and swap. Following my guide to RAID
superblock formats I used version
1.0 format for the /
array, since I'm going to end up with /boot
on it. Having created them as /dev/md10
and /dev/md11
it was
time to put them in /etc/mdadm.conf
. The most convenient way is
to use 'mdadm --examine --scan
' and then copy the relevant output
into mdadm.conf
by hand. Once you have updated mdadm.conf
, you
also need to update the initramfs version of it by rebuilding the
initramfs. Although you can do this for all kernel versions, I prefer
to do it only for the latest one so that I have a fallback path if
something explodes. So:
dracut --kver $(uname -r) --force
(This complained about a broken pipe for cat
but everything seems
to have worked.)
When I created the new RAID arrays, I took advantage of an mdadm
feature to give them a name with -N
; in particular I named them
'ssd root' and 'ssd swap'. It turns out that mdadm --examine --scan
tries to use this name as the /dev/
name of the array and the
initramfs doesn't like this, so on boot my new arrays became md126
and md127, instead of the names I wanted. To fix that I edited
mdadm.conf
to give them the proper names, and while I was there I
added all of the fields that my other (much older) entries had:
ARRAY /dev/md10 metadata=1.0 level=raid1 num-devices=2 UUID=35d6ec50:bd4d1f53:7401540f:6f971527 ARRAY /dev/md11 metadata=1.2 level=raid1 num-devices=2 UUID=bdb83b04:bbdb4b1b:3c137215:14fb6d4e
(Note that specifying the number of devices may have dangerous consequences if you don't immediately rebuild your initramfs. It's quite possible that Fedora 25 would have been happy without it, but I don't feel like testing. There are only a finite number of times I'm interested in rebooting my home machine.)
After copying my root filesystem from its old home on SATA HDs to
the new SSD filesystem, there were a number of changes I need to
make to actually use it (and the SSD-based swap area). First, we
modify /etc/fstab
to use the UUIDs of the new filesystem and swap
area for /
and, well, swap. The easiest way to get these UUIDs
is to use blkid
, as in 'blkid /dev/md10
' and 'blkid /dev/md11
'.
(For now I'm mounting the old HD-based root filesystem on /oldroot
,
but in the long run I'm going to be taking out those HDs entirely.)
But we're not done, because we need to make some GRUB2 changes in
order to actually boot up with the new root filesystem. A normal
kernel boot line in grub.cfg
looks like this:
linux /vmlinuz-4.9.9-200.fc25.x86_64 root=UUID=5c0fd462-a9d7-4085-85a5-643555299886 ro acpi_enforce_resources=lax audit=0 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us rd.md.uuid=d0ceb4ac:31ebeb12:975f015f:1f9b1c91 rd.md.uuid=c1d99f17:89552eec:ab090382:401d4214 rd.md.uuid=4e1c2ce1:92d5fa1d:6ab0b0e3:37a115b5 rootflags=rw,relatime,data=ordered rootfstype=ext4
This specifies two important things, the UUID of the root filesystem
in 'root=...
' and the (software RAID) UUIDs of the software RAID
arrays that the initramfs should assemble in early boot in the
'rd.md.uid=...
' bits (per the dracut.cmdline
manpage, and
also). We need to change the root filesystem
UUID to the one we've already put into /etc/fstab
and then add
rd.md.uuid=
settings for our new arrays. Fortunately mdadm
has
already reported these UUIDs for us and we can just take them from
our mdadm.conf
additions. Note that these two UUIDs are not the
same; the UUID of a filesystem is
different than the UUID of the RAID array that contains it, and one
will (probably) not work in the place of the other.
(In the long run I will need to take out the rd.md.uuid
settings
for the old HD-based root and swap partitions, since they don't
need to be assembled in early boot and will actively go away someday.)
The one piece of the transition that's incomplete is that /boot
is still on the HDs. Migrating /boot
is somewhat more involved
than migrating the root filesystem, especially as I'm going to merge
it into the root partition when I do move it. In the past I've
written up two aspects of that move to cover the necessary
grub.cfg
changes and a BIOS configuration
change I'll need to make to really make my new SSDs into the BIOS
boot drives, but I've never merged /boot
into /
in the process of such a move and I'm sure there will be
new surprises.
(This is where I cough in quiet embarrassment and admit that even
on my work machine, which moved its /
filesystem to SSDs some
time ago, my /boot
still comes from HDs. I really should fix that
by merging /boot
into the SSD /
at some point. Probably I'll
use doing it at work as a trial run for doing it at home, because
I have a lot more options for recovery if something goes wrong at
work.)
PS: The obvious thing to do for merging /boot
into /
is to build
a Fedora 25 virtual machine with a separate /boot
and then use
it to test just such a merger. There's no reason to blow up my
office workstation when I can work out most of the procedure
beforehand. This does require a new custom-built Fedora 25 VM image,
but it's still probably faster and less hassle than hacking up my
office machine.
PPS: It's possible that grub2-mkconfig
will do a lot of this work
for me (even things like the rd.md.uuid
and root=
changes). But
I have an old grub.cfg
that I like and grub2-mkconfig
would
totally change it around. It's easier to hand modify grub.cfg
than write the new config to a new file and then copy bits of it,
and in the process I wind up with a better understanding of what's
going on.
|
|