Re: Disappointing speed with ZFS

Alexey Tarasov wrote:
> Hello.
> 
> I am trying to use ZFS to store my torrent downloads. I noticed that
> hashing in rtorrent works 10 times slower than the same disk with UFS.

I've done some extensive file system testing and here are my results
with bonnie++ for UFS+SU vs ZFS on AMD64, 6 GB RAM (1 GB for kmem), on a
RAID10 volume of 15 kRPM SAS drives:

UFS+SU: write: 109 MB/s, read: 111 MB/s, random file creation: 36500 f/s
ZFS: write: 95 MB/s, read: 180 MB/s (!!), random file creation: 40522 f/s

Read speed for ZFS seems too high to be valid, it's probably some cache
effects (though tests were done on a file more than twice the RAM size).
In any case, ordinary hashing should cause sequential reading, and these
seem really fast.

There could be one more thing: ZFS tries to write data sequentially,
like a log file system, and if the download was done in "parallel", many
pieces from different areas of the file at the same time (which is
normally the case for torrents), it might have gotten very fragmented on
the drive.

You can verify this by creating a similarily-sized ordinary file with dd
(the file should be large enough not to fit in the memory cache, or the
test should be done after a reboot) and then run iostat in one console
while reading the files (separately, one at a time, with dd or cat) in
another. A very fragmented file should have significantly higher tps count.