Sysconfig's Blog

To cut a long story short:

• massive speed improvement
• excellent for work with many small files (compile times for big projects reduced significantly, to give one example)
• good value for money (~ £125)

And now the down-sides:

• getting very hot

Yes, it has a reason that ZFS is not yet marked stable on FreeBSD! I had to learn the hard lesson today.

Under very high load and many concurrent read requests (I set up the company’s mail server with ZFS and root from ZFS), the two disks in the Raid array repeatedly lost sync, forcing an automatic re-silvering (auto healing) process to be started, which blocked the system as everything (except /boot) was running from that ZFS arrray. As far as I figured out, the system halted entirely as there was another inconsistency occuring while the re-silvering was still in progress.

I would have investigated further, if it wasn’t a crucial production machine. And that kind of traffic is very difficult to simulate under laboratory situations (maybe I can do that when I have more time). So I had to revert back to UFS as the downtime had to be minimized. It’s a shame, really, because I love the features ZFS offers. On my private server it runs very smoothly, but traffic, load and I/O are not comparable to the mail server in question.

Yesterday I set up one of our new storage machines for testing: Dell 2950, Quad Xeon, 8GB, 6×750 GB HDD. I installed FreeBSD 7 with ZFS (following up this article). Firstly it seemed to be a bit tricky, because the PERC/6i controller configuration is — sorry — crap from the usuability point of view. It seemed not to support non-RAID configurations, but taking a closer look it turned out to be a wrong assumption. Six RAID-0 Arrays with only one drive each is in fact the same as no RAID at all. (The reason why RAID does not make sense is, that ZFS will do this job, and its auto-healing is much better than any hardware controller’s auto-healing)

After having set up the minimal FreeBSD and doing some tuning (such as creating the ZFS volumes), I ran some tests. You won’t believe me, but writing a 10GB file (/dev/random to the ZFS volume) resulted in a transfer rate at about 160MB/sec and reading (cp testfile /dev/null) was done at a speed of more than 270MB/sec!!

To be continued…

As the company I work with has to store many media files, backups, rapidly changing documents and so on, they used to run a NetworkAppliance FAS2020 storage machine, which is quite neat. Unfortunately, the current setup does not allow to scale the volumes any more. So we needed to find an alternative.

Continue reading »