Wandering Thoughts archives

Converting filesystems from ext3 to ext4, and concerns attached to it (plus bad news for me)

Yesterday I covered why I basically need to move from ext3 to ext4 and I said that the mechanisms of this transition raise a few issues. So let's talk about them, and in the process I'll wind up with disappointing news for my own case. Actually, let's lead with the disappointing news:

An ext3 filesystem converted to ext4 almost certainly won't support ext4's nanosecond file timestamps; it will only have ext3 one-second ones.

On the surface the mechanics of converting from ext3 to ext4 are simple and relatively straightforward, with two levels. The first level is simply to mount filesystems as ext4 instead of ext3. This enables backwards-compatible filesystem features and is fully reversible (at least according to the sources I've read). As a result it ought to be as safe as ext4 itself, and these days that's pretty safe. However this only gives you features like delayed allocation (which some people consider a non-feature for complex reasons).

(Checksumming the journal may be one of the things you get here; it's not clear to me.)

To fully convert a filesystem to ext4, one must also use tune2fs to enable the ext4-specific filesystem features that affect the on disk format of the filesystem. Various sources (eg the ext4 wiki) say that this is:

tune2fs -O extents,uninit_bg,dir_index /dev/<WHAT>

(In theory dir_index is also an ext3 feature, but on the other hand at least some of my ext3 filesystems don't have it. They may have started out as ext2 filesystems.)

However, this is not the full set of feature differences between ext3 and ext4 today. The Fedora 21 /etc/mke2fs.conf adds huge_file, flex_bg, dir_nlink, and extra_isize, and the tune2fs manpage says that all of them can be set on an existing filesystem (although flex_bg and extra_isize probably have no meaningful effect). Per the tune2fs manpage and various instructions on ext3 to ext4 conversions, setting uninit_bg requires an e2fsck and setting dir_index is helped by doing an e2fsck -D.

Now we get to my concerns. The first concern is straightforward; at least some 'how to convert from ext3 to ext4' sources contain prominent 'back up your data just in case' warnings. These sources are also relatively old ones and I suspect that they date from the days when ext4 and this process was new and thus uncertain. Notably, the ext4 wiki itself doesn't have any such cautions these days. Still, it's a bit alarming.

The second concern is whether this is actually going to do me any good. My entire purpose for doing this is to get support for sub-second file timestamps, because that's basically the only user visible ext3 versus ext4 difference (and software is noticing). The existing documentation is not clear on whether ext4 automatically starts doing this on an existing ext3 filesystem, if you have to add the extra_isize feature, or if a converted ext3 filesystem simply can't do this because its existing (and future) inodes are too small to hold the extra data.

Unfortunately the news here is not good. Additional sources (this description of timestamps in ext4, this SA answer) say that the high resolution timestamps definitely go in the extra space in normal ext4 inodes, space that my ext3 filesystem inodes just don't have (ext3 defaults to 128 byte inodes, ext4 to 256 byte ones). Converting a filesystem from ext3 to ext4 does not enlarge its inodes, which means that a normal ext3 filesystem converted to ext4 will never support high resolution timestamps.

Given this, there's not much point in putting myself through this in-place conversion effort (in fact converting my filesystems would be silently misleading). The only way I'd ever get better timestamps would be to make new ext4 filesystems from scratch, copy all the current data into them, and then replace my existing filesystem mounts with mounts of the new ext4 filesystems. Among other things, this is a pain in the rear.

(It's achievable, at least in theory, as my LVM pool has plenty of unused space. It would mean that I'd only 'convert' a couple of filesystems, the ones most likely to run into high resolution file timestamp issues.)

PS: You can see how big your ext3 inodes are with 'tune2fs -l /dev/<WHAT>'; it's reported near the end. This will also let you see what features are set on your filesystem (and thus let you know that your ext3 filesystems are without dir_index).

Why I'm interested in converting my ext3 filesystems to ext4

My home machine has a sufficiently old set of filesystems that many of my actively used filesystems are still ext3, not ext4, including both my home directory and where I keep code. Normally this isn't something that I particularly think or worry about; it's not like ext4 is a particularly radical advance from ext3 (certainly not the same sort of jump that was ext2 to ext3, where you got fast crash recovery). As a sysadmin I'm generally cautious with filesystem choices anyways (at least when I'm not being radical); I used ext2 over ext3 for years after the latter came out, for example, on the principle that I'd let other people find the problems.

It turns out that there is one important thing that ext4 has and ext3 does not: ext4 has sub-second file timestamps, while ext3 only does timestamps to the nearest second. Modern machines are fast enough that nearest second timestamps are increasingly not really good enough when building software or otherwise doing things that care about relative file timestamps and 'is X more recent than Y'. Oh, sure, it works most of the time, but every so often things go wrong or you find assumptions buried in other people's software.

Most people don't notice these things because most people are now using filesystems that support sub-second file timestamps (which is almost all modern Linux filesystems). What this tells me is that I'm increasingly operating in an unusual and effectively unsupported environment by continuing to use ext3. As time goes by, more and more software is likely to assume sub-second file timestamps basically by default (because the authors have never run it on a system without them) and not work quite right in various ways. I can fight a slow battle against what is effectively a new standard of sub-second file timestamps, or I can give in and convert my ext3 filesystems to ext4. It's not like ext4 is exactly a new filesystem these days, after all (Wikipedia dates it to 2008).

The mechanics of this conversion raise a few issues, but that's something for another entry.