Re: processes freezing when writing to gstripe'd device

> +> I am observing processes performing operations on a gstripe device
> +> freeze in state 'bufwait'. An 'rm' process is stuck right now. The rest
> +> of the system is fine.
> +> 
> +> What's the best way to look in to this? I can't attach to rm with gdb
> +> (it just ends up waiting for something). I can drop to kdb, but have no
> +> idea where to go from there.
> 
> You could use 'ps' command from DDB to which processes are alseep.
> Then you can run 'tr <PID>' where <PID> is PID of sleeping process.
> Look for processes related somehow to this problem.
> 
> It'll be also great if you can provide exact procedure which will also
> me to reproduce this problem.

Okay, I updated to current as of yesterday and still seeing the same
problem. I'm new to these bits of the kernel but it looks like a locking
problem. This is what I am doing:

dd if=/dev/zero of=sd0 count=20480
cp sd0 sd1
mdconfig -a -t vnode -f sd0
mdconfig -a -t vnode -f sd1
gstripe label bork md0 md1
newfs /dev/stripe/bork
mkdir teststripe
mount /dev/stripe/bork teststripe
cd teststripe

Now I repeatedly 'cvs checkout' and 'rm -rf' the FreeBSD src tree.
Usually it freezes during the first checkout.

Siginfo shows:

load:1.14  cmd: cvs 801 [biowr] 0.33u 3.35s 14% 2840k

A trace of the frozen cvs process 801 shows:

KDB: enter: manual escape to debugger
[thread 100006]
Stopped at      kdb_enter+0x2b: nop
db> tr 801
sched_switch(c1a87580,0) at sched_switch+0x12b
mi_switch(1,0) at mi_switch+0x24d
sleepq_switch(c63ee500,d0c83814,c06030e9,c63ee500,0) at
sleepq_switch+0xe0
sleepq_wait(c63ee500,0,0,0,c07f3ab7) at sleepq_wait+0xb
msleep(c63ee500,c08ddd80,4c,c07f40d1,0) at msleep+0x375
bwait(c63ee500,4c,c07f40d1) at bwait+0x47
bufwait(c63ee500,c088f1a0,c1cd6318,c63ee500,0) at bufwait+0x2d
ibwrite(c63ee500,d0c838d8,c071906e,c63ee500,a00) at ibwrite+0x3e2
bwrite(c63ee500,a00,0,ee,c19b1834) at bwrite+0x32
ffs_update(c19c3738,1,0,c19b808c,c19c3738) at ffs_update+0x302
ufs_makeinode(81a4,c199f840,d0c83bf8,d0c83c0c) at ufs_makeinode+0x3a3
ufs_create(d0c83a74,d0c83b30,c0655238,d0c83a74,c08b8c00) at
ufs_create+0x26
ufs_vnoperate(d0c83a74) at ufs_vnoperate+0x13
vn_open_cred(d0c83be4,d0c83ce4,1a4,c1d47700,8) at vn_open_cred+0x174
vn_open(d0c83be4,d0c83ce4,1a4,8,c08ad240) at vn_open+0x1e
kern_open(c1a87580,8199430,0,602,1b6) at kern_open+0xd2
open(c1a87580,d0c83d14,3,1be,292) at open+0x18
syscall(2f,bfbf002f,bfbf002f,8,2836c7f8) at syscall+0x217
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (5, FreeBSD ELF32, open), eip = 0x282e2437, esp =
0xbfbfdd7c, ebp = 0xbfbfdda8 ---

If I try to rm -rf src, that process also ends up waiting:

sched_switch(c1a879a0,0) at sched_switch+0x12b
mi_switch(1,0) at mi_switch+0x24d
sleepq_switch(c199f8ec,d0c8ca58,c06030e9,c199f8ec,0) at
sleepq_switch+0xe0
sleepq_wait(c199f8ec,0,0,0,c199f840) at sleepq_wait+0xb
msleep(c199f8ec,c08ad898,50,c07f34e1,0) at msleep+0x375
acquire(d0c8cab0,1000040,600,c1a879a0,0) at acquire+0x9e
lockmgr(c199f8ec,1010002,c199f840,c1a879a0,10002) at lockmgr+0x37e
ufs_lock(d0c8cadc,d0c8caf8,c06563a5,d0c8cadc,c088f0a0) at ufs_lock+0x3c
ufs_vnoperate(d0c8cadc) at ufs_vnoperate+0x13
vn_lock(c199f840,10002,c1a879a0,c1cda000,c1cda000) at vn_lock+0xc5
vget(c199f840,2,c1a879a0,139a,c1a879a0) at vget+0xc9
vfs_cache_lookup(d0c8cbb4,d0c8cbd0,c0646613,d0c8cbb4,c1a879a0) at
vfs_cache_lookup+0x1c9
ufs_vnoperate(d0c8cbb4) at ufs_vnoperate+0x13
lookup(d0c8cc30,c1a879a0,c197d948,0,c07f51e9) at lookup+0x2cf
namei(d0c8cc30,8051aa8,0,0,c16e2c60) at namei+0x1f0
lstat(c1a879a0,d0c8cd14,2,7,292) at lstat+0x4a
syscall(2f,2f,2f,8051a00,8051a48) at syscall+0x217
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (190, FreeBSD ELF32, lstat), eip = 0x280cb757, esp =
0xbfbfe74c, ebp = 0xbfbfe7e8 ---

I'll keep poking around - if you have any further suggestions or need
other information, fire away.

Regards
Sam.