Re: ZFS kmem_map too small.

On Tue, Oct 09, 2007 at 12:41:21PM -0700, Darren Reed wrote:
> Pawel Jakub Dawidek wrote:
> >Here are some updates:
> >
> >I was able to reproduce the panic by rsyncing big files and trying
> >bonnie++ test suggested in this thread.
> >
> >Can you guys retry with this patch:
> >
> >	http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch
> >  
> 
> So, I have a question...
> What happens if the "for (i = 0..)" is changed to "while(1)" and
> the "panic" is subsequently removed?

I think it should stay to give the user a hint what's going on instead
of hanging there forever.

> It appears like the code changes the meaning of "WAIT" to "wait
> for 4 seconds" then panic if it won't work.  Previously, "WAIT" was
> not waiting at all...whch could be described as a bug!

It's actually 7 seconds:)

> If I recall correctly, ZFS caches writes and doe them in spurts and
> that those spurts are spaced out more than 4 seconds.  (For the
> curious, do "zpool status" and observe the gap in time between
> write activity.)
> 
> If you start a large amount of I/O, it is possible that all the KVA will
> be used up and ZFS will not get a chance to flush its buffers before
> the 4s timer here expires.  Does that sound plausible?

It depends if the problem we see is because of caching/delaying writes
or just caching data for faster reads. If the latter, the cache can be
just thrown away, so it's much faster than waiting for buffers to be
flushed in former case. ZFS flushes buffers every 5 seconds by default
or when there is too much data, so 7 seconds sounds reasonable.

> Would doubling the 8 to (say) 16 be beneficial here, to at least make
> the waiting span one ZFS flush out to disk?

Note that this is visible by the user as almost complete system hang, I
think. 16 would make it to wait for 30 seconds.
I do agree that waiting even 30 seconds in some extremly rare situations
is better than panicing, but I'd first see if 8 fixes the problem.

In my testing kernel I added debug printf to see when 'i' is larger than
0 - every value larger than 0 means panic with the old kernel.  I never
observed 'i' larger than 1.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd_at_FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!