Why Linux needs ZFS and badly

[mehulved]

An interesting article by a gentoo developer, featured on planet gentoo.

[before you flame me, I know that Linux (Gentoo, in fact) has zfs-fuze, but it is still pretty experimental, and it runs in user space, which makes it noticeably slower]
ZFS is Sun's very cool filesystem. I won't go into detail here - just google it - but it has some eye-opening features, the most critical of which is end-to-end data integrity. Unfortunately, ZFS's license is incompatible with the GPL.
I say "critical" because I have a strong feeling that silent data corruption is far more prevalent than most people believe. Also, I just don't buy the argument that bit-for-bit reliability is only important for servers. Yes, in certain circumstances, a bit flip here or there may not be noticed, but I think that is scary as hell. Personally, I'd rather know; I count on computers to copy the bits exactly, don't you? We simply cannot tolerate random bit errors, no matter how "unnoticeable". And you will notice if that bit flip hits a critical part of your file.
With disk drives becoming larger and larger and the marketing departments of drive manufacturers knowing that the general public doesn't understand these issues, they tend to boast speed and size over reliability. We will soon be in real trouble. For an upcoming space mission I'll be working on at my job, we may have to buy petabytes of storage. With this much, the current hard drive uncorrectable error rates will cause multiple errors per day, letting the data potentially bit rot with current modern filesystems. And just as bad, swap space is also susceptible. So even if you have ECC memory (and I recommend it highly), if your data ends up in swap, you are vulnerable.
In my experience with computers, I have caught two examples of silent data corruption. These are ones I actually discovered. It freaks me out to think there may be many more that went unnoticed. And both were due to bad IDE cables (so even the hard disk error rates don't count here) on two different computer systems. The first on the old and slow PATA and was some data pattern dependent copy glitch, where a diff found the problem. The other was this past year on a modern UDMA/80-conductor cable, and it was found by ZFS - it appears that during some reported DMA errors (probably the cable's fault), a 64K file block got written to the wrong spot on the disk (PATA does not protect the data address part of the communication).
ZFS is the only filesystem that actually will catch silent corruption in the whole chain: ATA interface -> cable -> disk (HW and firmware). For those who say, "Why not RAID?", well, RAID will save you if a whole drive fails, but not these more insidious issues. I bet Linus and others are seriously thinking about what to do, since what once was considered rare could become commonplace. There are rumors Apple will adopt ZFS, and FreeBSD already has it in its kernel (and, of course, Solaris has it). For now, zfs-fuse is very interesting, but I think we need such protection of our data in the kernel, and soon.

Source:- http://planet.gentoo.org/developers/..._zfs_and_badly



Do check the planet too, there are some interesting comments that follow the article too.

[ray|raven]

^ I read about ZFS on LinuxFormat some time back and was freakin shocked to see the limits.
And , btw have you tried it in FUSE mode?

[praka123]

Ermm..ext4 will hopefully cover all these problemos?

[mehulved]

I don't use linux. And zfs is natively supported by FreeBSD. But, I had formatted my partitions to ufs before considering about zfs.

[kalpik]

^^ Why dont you use ZFS now?

[ray|raven]

@praka
Dude, Read about ZFS ,
The features it got are freakin awesome,

here's a few :

264 — Number of snapshots of any file system[8]
248 — Number of entries in any individual directory[9]
16 EiB (264 bytes) — Maximum size of a file system
16 EiB — Maximum size of a single file
16 EiB — Maximum size of any attribute
256 ZiB (278 bytes) — Maximum size of any zpool
256 — Number of attributes of a file (actually constrained to 248 for the number of files in a ZFS file system)
256 — Number of files in a directory (actually constrained to 248 for the number of files in a ZFS file system)
264 — Number of devices in any zpool
264 — Number of zpools in a system
264 — Number of file systems in a zpool

We gotta pass through quite a lot of decades before we end up at the limits IMO.

[mehulved]

Originally Posted by kalpik

^^ Why dont you use ZFS now?

You can't do it AFAIK.

[kalpik]

Hmm.. lemme try FreeBSD 7.0 in VMWare

[mehulved]

No. try gentoo/freebsd-7.0

[kalpik]

^^ Wuzzat? :s

[praka123]

^heh!Debian/kfreebsd?

[MetalheadGautham]

awssome one! but does sabayon support zfs, if gentoo does ?
and I think ext4 is going to try hard to implement zfs features.
poor me. I am still stuck with good old standard ext3

PS: it amuses me to look at the way we BSD/Solaris/Linux boys are thinking of high end data safety while the windows boys are STILL happy with NTFS

edit: no offence. please don't take flame.

[ray|raven]

^Dude, dont convert this thread to another win vs lin discussion.
Please.

[MetalheadGautham]

Originally Posted by rayraven

^Dude, dont convert this thread to another win vs lin discussion.
Please.

actually, its lin vs solaris/bsd.

[mehulved]

Originally Posted by kalpik

^^ Wuzzat? :s

Gentoo is just a set of utilities not a distro by itself, as the popular belief goes. So, gentoo linux is just one part of gentoo. Similarly, there's gentoo freebsd, that's freebsd kernel and userland tools like libraries, pam, etc but gentoo portage and tools.

Originally Posted by MetalheadGautham

awssome one! but does sabayon support zfs, if gentoo does ?

No, the support is in the kernel. Sabayon is based on gentoo linux, ie it uses linux kernel. Linux kernel still doesn't support zfs natively, like it's mentioned in the article. So, the answer is no, sabayon cannot support zfs.
It's gentoo/*bsd that can support zfs. Basically it's the BSD kernels that support zfs.

[kalpik]

Ah ok.. Thanks.. Didnt know that So does Gentoo/FreeBSD support ZFS? Im more interested in ZFS than Gentoo/FreeBSD.. Thinking of installing Solaris now

[mehulved]

Originally Posted by kalpik

Ah ok.. Thanks.. Didnt know that So does Gentoo/FreeBSD support ZFS? Im more interested in ZFS than Gentoo/FreeBSD..

Yes. But go for version 7.0 of FreeBSD.