== Some notes on moving a software RAID-1 root filesystem around (to SSDs) A while ago I got some SSDs for [[my kind of old home machine HomeMachine2011]] but didn't put them to immediate use for [[various reasons PlanningHomeSSDShuffle]]. Spurred on first by the feeling that I should get around to it sometime, before my delay got too embarrassing, and then by [[one of my system drives https://twitter.com/thatcks/status/833482602556297216]] apparently going into [[slow IO mode ../tech/QuietSATADegradation]] for a while, I've now switched my root filesystem over to my new SSDs. I've done this process before, but this time around I want to write down notes for my future reference rather than having to re-derive all the steps each time. All of this is primarily for Fedora, currently Fedora 25; some steps will differ on other distributions such as Ubuntu. I partitioned using [[GPT partitions LinuxGPTAndEFI]], not particularly because I needed to with 750 GB SSDs but because it seemed like a good idea. I broadly copied the partitioning I have on my SSDs at work for no particularly strong reason, which means that I set it up this way: > |_. Number | Size | Code | Name > | 1 | 256 MB | EF00 | EFI System > | 2 | 1 MB | EF02 | BIOS boot partition > | 3 | 100 GB | FD00 | Linux RAID > | 4 | 1 GB | FD00 | Linux RAID (swap) > | 5 | | BF01 | ZFS Some of this is likely superstition by now, such as the BIOS boot partition. With the pair of SSDs partitioned, I set up the software RAID-1 arrays for the new _/_ and swap. Following [[my guide to RAID superblock formats SoftwareRaidSuperblockFormats]] I used version 1.0 format for the _/_ array, since I'm going to end up with _/boot_ on it. Having created them as _/dev/md10_ and _/dev/md11_ it was time to put them in _/etc/mdadm.conf_. The most convenient way is to use '_mdadm --examine --scan_' and then copy the relevant output into _mdadm.conf_ by hand. Once you have updated _mdadm.conf_, you also need to update the initramfs version of it by rebuilding the initramfs. Although you can do this for all kernel versions, I prefer to do it only for the latest one so that I have a fallback path if something explodes. So: .pn prewrap on > dracut --kver $(uname -r) --force (This complained about a broken pipe for _cat_ but everything seems to have worked.) When I created the new RAID arrays, I took advantage of an _mdadm_ feature to give them a name with _-N_; in particular I named them 'ssd root' and 'ssd swap'. It turns out that _mdadm --examine --scan_ tries to use this name as the _/dev/_ name of the array and the initramfs doesn't like this, so on boot my new arrays became md126 and md127, instead of the names I wanted. To fix that I edited _mdadm.conf_ to give them the proper names, and while I was there I added all of the fields that my other (much older) entries had: > ARRAY /dev/md10 metadata=1.0 level=raid1 num-devices=2 UUID=35d6ec50:bd4d1f53:7401540f:6f971527 > ARRAY /dev/md11 metadata=1.2 level=raid1 num-devices=2 UUID=bdb83b04:bbdb4b1b:3c137215:14fb6d4e (Note that specifying the number of devices [[may have dangerous consequences if you don't immediately rebuild your initramfs RaidGrowthGotcha]]. It's quite possible that Fedora 25 would have been happy without it, but I don't feel like testing. There are only a finite number of times I'm interested in rebooting my home machine.) After copying my root filesystem from its old home on SATA HDs to the new SSD filesystem, there were a number of changes I need to make to actually use it (and the SSD-based swap area). First, we modify _/etc/fstab_ to use the UUIDs of the new filesystem and swap area for _/_ and, well, swap. The easiest way to get these UUIDs is to use _blkid_, as in '_blkid /dev/md10_' and '_blkid /dev/md11_'. (For now I'm mounting the old HD-based root filesystem on _/oldroot_, but in the long run I'm going to be taking out those HDs entirely.) But we're not done, because we need to make some GRUB2 changes in order to actually boot up with the new root filesystem. A normal kernel boot line in _grub.cfg_ looks like this: > linux /vmlinuz-4.9.9-200.fc25.x86_64 root=UUID=5c0fd462-a9d7-4085-85a5-643555299886 ro acpi_enforce_resources=lax audit=0 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us rd.md.uuid=d0ceb4ac:31ebeb12:975f015f:1f9b1c91 rd.md.uuid=c1d99f17:89552eec:ab090382:401d4214 rd.md.uuid=4e1c2ce1:92d5fa1d:6ab0b0e3:37a115b5 rootflags=rw,relatime,data=ordered rootfstype=ext4 This specifies two important things, the UUID of the root filesystem in '_root=..._' and the (software RAID) UUIDs of the software RAID arrays that the initramfs should assemble in early boot in the '_rd.md.uid=..._' bits (per the [[_dracut.cmdline_ manpage http://man7.org/linux/man-pages/man7/dracut.cmdline.7.html]], [[and also DracutNeededArguments]]). We need to change the root filesystem UUID to the one we've already put into _/etc/fstab_ and then add _rd.md.uuid=_ settings for our new arrays. Fortunately _mdadm_ has already reported these UUIDs for us and we can just take them from our _mdadm.conf_ additions. Note that [[these two UUIDs are not the same IDsForDisksAndFilesystems]]; the UUID of a filesystem is different than the UUID of the RAID array that contains it, and one will (probably) not work in the place of the other. (In the long run I will need to take out the _rd.md.uuid_ settings for the old HD-based root and swap partitions, since they don't need to be assembled in early boot and will actively go away someday.) The one piece of the transition that's incomplete is that _/boot_ is still on the HDs. Migrating _/boot_ is somewhat more involved than migrating the root filesystem, especially as I'm going to merge it into the root partition when I do move it. In the past I've written up two aspects of that move to cover [[the necessary _grub.cfg_ changes Grub2MoveBootDisk]] and [[a BIOS configuration change I'll need to make to really make my new SSDs into the BIOS boot drives GrubDiskMismatchError]], but I've never merged _/boot_ into _/_ in the process of such a move and I'm sure there will be new surprises. (This is where I cough in quiet embarrassment and admit that even on my work machine, which moved its _/_ filesystem to SSDs some time ago, my _/boot_ still comes from HDs. I really should fix that by merging _/boot_ into the SSD _/_ at some point. Probably I'll use doing it at work as a trial run for doing it at home, because I have a lot more options for recovery if something goes wrong at work.) PS: The obvious thing to do for merging _/boot_ into _/_ is to build a Fedora 25 virtual machine with a separate _/boot_ and then use it to test just such a merger. There's no reason to blow up my office workstation when I can work out most of the procedure beforehand. This does require a new custom-built Fedora 25 VM image, but it's still probably faster and less hassle than hacking up my office machine. PPS: It's possible that _grub2-mkconfig_ will do a lot of this work for me (even things like the _rd.md.uuid_ and _root=_ changes). But I have an old _grub.cfg_ that I like and _grub2-mkconfig_ would totally change it around. It's easier to hand modify _grub.cfg_ than write the new config to a new file and then copy bits of it, and in the process I wind up with a better understanding of what's going on.