Re: Low NTFS read performance

Diomidis Spinellis wrote in freebsd-stable_at_:
> I can't get an Ultrium-2 LTO drive to stream, and (I think) I've traced 
> the problem to the read performance of the USB2-attached NTFS disk, and 
> specifically the NTFS filesystem.  I'm reading a single 190GB file, and 
> the throughput I'm getting is 5.4MB/s:
> 
> $ dd if=ad2c.dump of=/dev/null bs=1M
> ^Τ
> load: 0.04  cmd: dd 1434 [biord] 0.00u 4.78s 2% 1672k
> 610+0 records in
> 610+0 records out
> 639631360 bytes transferred in 117.937613 secs (5423472 bytes/sec)
> 
> The is an old but relatively fast machine
> 
> CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2407.18-MHz 686-class CPU)
> 
> running 7.0-RC1:
> 
> $ uname -a
> FreeBSD icarian.dmst.aueb.gr 7.0-RC1 FreeBSD 7.0-RC1 #0: Mon Dec 24 
> 12:18:24 UTC 2007 
> root_at_logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
> 
> and the load during the read remains comfortably low:
> 
> $ uptime
>  6:34PM  up  3:50, 3 users, load averages: 0.09, 0.04, 0.04
> 
> Reading from the raw device tripples the performance:
> 
> # dd if=/dev/da0s1 bs=1m of=/dev/null
> 533725184 bytes transferred in 34.777460 secs (15346871 bytes/sec)
> 
> bringing it on par with what I get from Windows (on a different machine):
> 
> F:\>dd if=other.20051007.tgz of=/dev/null bs=1M
> 1231030919 bytes (1.2 GB) copied, 82.845 s, 14.9 MB/s
> 
> [...]
> 
> I'd appreciate any suggestions you may have.

Today I upgraded the machine to  8.0-CURRENT, and obtained similar results:
-  5372041 bytes/sec reading an NTFS file
- 15369758 bytes/sec reading from the raw device

The raw/FS read performance difference with UFS is a lot lower:
- 33162654 bytes/sec reading a UFS file
- 47221133 bytes/sec reading from the raw device

I'd welcome any ideas on why the NTFS layer makes reading three time 
slower than reading from the raw device and suggestions for improving 
the performance.  I'd also be happy to work on improving the situation, 
if I get any pointers on where to start.

Diomidis Spinellis - http://www.spinellis.gr