ZFS not caching right?

Hi, it's me again with another stupid question,

ever since I switched my /usr and /home to ZFS, the system has become
_very_ unresponsive under load. This is not because of CPU load, but I/O
load.

This is on laptop with a lousy 4800RPM hard disk and 1 GB RAM. After
boot, the system is snappy as ever up to a certain point. If I build a
large port for example it takes less than an hour to bring the system to
a crawl.

Right now, I'm building OOo in the background and tried to start amarok.
This took, like 3 minutes or more while the disk is going nuts

# zpool iostat 1
tank        12.4G  14.8G    100      0  12.4M      0
tank        12.4G  14.8G     47     42  5.45M   387K
tank        12.4G  14.8G     27     85  3.34M   392K
tank        12.4G  14.8G     38     27  4.83M   129K
tank        12.4G  14.8G     47     46  5.94M   864K
tank        12.4G  14.8G     43     47  5.37M   242K
tank        12.4G  14.8G     84      0  10.2M      0
tank        12.4G  14.8G     63      0  7.92M      0
tank        12.4G  14.8G     87      0  10.9M      0
tank        12.4G  14.8G     73      0  9.16M      0
tank        12.4G  14.8G     35     72  4.46M   678K
tank        12.4G  14.8G     19     73  2.29M   997K
tank        12.4G  14.8G     71      3  8.91M  3.96K
tank        12.4G  14.8G     91      0  11.4M      0
tank        12.4G  14.8G     84      0  10.5M      0
tank        12.4G  14.8G     54     88  6.81M   403K
tank        12.4G  14.8G     77      0  9.65M      0
tank        12.4G  14.8G     82      0  10.3M      0

The mem and cache settings as shown by systat -vm are as follows

    7 users    Load  1.01  1.12  1.15                  Sep  6 20:42

Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP PAGER
        Tot   Share      Tot    Share    Free           in   out     in   out
Act  515148   55612  1283636   118028  138308  count                  6
All  713292   62660  3724916   139796          pages           2     27
Proc:                                                            Interrupts
  r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt    322 cow     370 total
  4       1 143      3740 6830  20k  371  371 5797   4092 zfod    100 clk irq0
                                                       58 ozfod       atkbd0 1
13.0%Sys   0.3%Intr  6.7%User 74.9%Nice  5.1%Idle       1%ozfod   128 rtc irq8
|    |    |    |    |    |    |    |    |    |    |       daefr       pcm0 iwi0+
=======>>>-------------------------------------      4016 prcfr   106 cbb0 bfe0+
                                         3 dtbuf     4824 totfr    28 ata0 irq14
Namei     Name-cache   Dir-cache     50000 desvn        4 react     8 ata1 irq15
   Calls    hits   %    hits   %     30904 numvn          pdwak       psm0 irq12
   38614   38555 100                 17491 frevn          pdpgs
                                                        6 intrn
Disks   ad0   da0 pass0                            196884 wire
KB/t   8.36  0.00  0.00                            552976 act
tps      27     0     0                            131128 inact
MB/s   0.22  0.00  0.00                             22796 cache
%busy     9     0     0                            115512 free
                                                   110176 buf


Yesterday, the cache value was hovering at 900 and not improving much,
this is abnormal, right? I "tuned" ZFS with the following settings as
recommended by the Wiki and on -current

vfs.zfs.zil_disable="1"
vfs.zfs.prefetch_disable="1"
vfs.zfs.arc_max="128*1024*1024"

Is the tremendous amount of disk reading due to ZFS' nature or is it VM
related?

Cheers,
Ulrich Spoerlein
-- 
It is better to remain silent and be thought a fool,
than to speak, and remove all doubt.