Deduplication and versioning file systems on Linux

published Oct 06, 2018 01:05 by admin ( last modified Oct 06, 2018 01:07 )

Summary: NILFS is the only one regarded as stable of the one's included in kernel in Linux. ZFS has a long history now also on Linux (outside of the kernel), RedHat is developing stratis and there is also bcachefs under development. RedHat is dropping support for btrfs. The author of bcacheFS claims Btrfs has low code quality. LVM can do deduplication if configured into thin pools.

Deduplicating file systems can create snapshots which on a deduplicating file system could just be folders. Two file system that can do snapshots and that are regarded as stable are:

ZFS— Has license problems on Linux, but it is possible to use. Needs 8GB of RAM or thereabouts for efficient deduplication.
NILFS—Regarded as stable on Linux, according to Wikipedia: "In this list, this one is [the] only stable and included in mainline kernel.". According to NILFS FAQ it does not support SELinux that well: "At present, NILFS does not fully support SELinux since extended attributes are not implemented yet

Another way of doing snapshots seems to be on a slightly higher level:

LVM thin pools—I guess this could work with any file system RedHat has some documentation here.

RedHat has decided not to support btrfs in the future and are working on something called Stratis, which is similar to LVM thin pools, built ontop of the XFS file system.

We may also get an alternative to btrfs on Linux with bcacheFS.