Re: Strange behaviour

From: Lachlan O'Dea <odela01_at_ca.com>
Date: Thu, 22 Jan 2004 12:54:48 +1100
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 22 Jan 2004, at 05:43, Kris Kennaway wrote:

> On Wed, Jan 21, 2004 at 01:39:59PM -0500, Ken Smith wrote:
>
>> I have the same machine (Dell 2650) and it's getting locked up in
>> a very similar way, you don't need to get NFS involved to have
>> processes get locked uup in getblk.  I'm slowly trying to remove
>> variables but so far it seems like network activity of some sort
>> helps cause the lockup.  The easiest way to make it lock up was
>> doing backups through the network.  But find's cranked up by the
>> nightly cron jobs can get locked in getblk as well (while there
>> are no NFS partitions mounted, but things like cvsup updates of
>> a local repo are happening).  Once things start to get locked up
>> like this the system slowly degrades.  I can usually ssh in and
>> reboot it if I catch it soon enough, if I leave it for a couple
>> of days it will seem like it's up (rwhod is running) but ssh-ing
>> in won't work.
>>
>> sledge (amd64 machine in the cluster) was showing similar symptoms
>> this morning, it had failed doing its nightly rebuild/reboot and
>> things like mtree commands were wedged since a day or two ago.
>>
>> The Dell I have here is not really in production at all, if me
>> doing anything here will help I'm game...
>
> OK, sounds like it might be something to do with vnode locking.
> Turning on DEBUG_VFS_LOCKS might help to find it.

I have exactly the same behaviour on a Dell PowerEdge 2500, including  
progressive degradation as more processes get hung on filesystem  
access. I don't need network activity to recreate it. Makewhatis at the  
end of installworld in single user mode all on the same local partition  
will hang forever on getblk. The find commands run by the daily cron  
jobs almost always hang as well. It seems to be triggered by certain  
patterns of filesystem use. Please see:

<http://docs.freebsd.org/cgi/mid.cgi?D90B8DC2-471E-11D8-A84C 
- -000A95DBB47C>

and I believe this thread is discussing the same thing:

<http://docs.freebsd.org/cgi/mid.cgi?7mznedas57.wl> (includes some DDB  
output)

I have DEBUG_LOCKS in my kernel, but I'm not really sure what to do  
with it. I haven't heard of DEBUG_VFS_LOCKS before, I'll look into it.

- --
Lachlan O'Dea
Computer Associates
Programmer
tel: +61 3 8416 5627
fax: +61 3 8416 5810
mobile: +61 412 390 650
odela01_at_ca.com

Relax and enjoy your shoes!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (Darwin)

iD8DBQFADy1ooBZahcUBONoRAh0cAJ9vS14txHvU+bLTP9bVs6CzUdQvDgCgpKXZ
1mXFjxGwTHMTij9ngeuVU2U=
=qNyc
-----END PGP SIGNATURE-----
Received on Wed Jan 21 2004 - 16:54:50 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:39 UTC