Re: FreeBSD 5.1-Release freezes

On Wed, 25 Jun 2003, Socketd wrote:

> I have just installed 5.1 Release on my laptop (6gb harddisk and 60 mb
> ram). I only installed the base system + man pages and the port
> collection. A little info:  / 128M Swap 256M /var 128M /var/tmp 256M
> (/tmp is symlinked to this)  /home 256M /usr The rest
> 
> I compiled a new kernel, with no support for SCSI, RAID, IPv6 and all
> but my USB NIC (also removed a few USB things). I added:  options
> SC_DISABLE_REBOOT options MAC device pcm
> 
> It compiled fine and I installed it. Now, if I run with this kernel the
> system freezes! I can boot, login and run a few commands, but that's it. 
> There is nothing in the logs, but when I tried ssh'ing to the mashine I
> got this before it froze:  fork() - Not enough ressources available at
> this time
> 
> (or something like that). 
> 
> Is there a memory leak somewhere or am I doing something wrong? (I
> compiled and installed the kernel with:
> make buildkernel KERNCONF=SOCKETD
> make installkernel KERNCONF=SOCKETD)
> 
> btw please cc to me as I am not on the list. 

Sounds like a memory leak to me; it might also be a locking problem.  Are
you actually running with any MAC policies, or just with the framework? 

The code most likely to cause a memory leak in the MAC Framework is the
label management code, since that's the only code that really does much in
the way of memory allocaiton.  Try compiling options MAC_DEBUG into your
kernel, which causes the MAC Framework to track the number of labels it
has allocated/free'd in a series of variables: 

static unsigned int nmacmbufs, nmaccreds, nmacifnets, nmacbpfdescs,
    nmacsockets, nmacmounts, nmactemp, nmacvnodes, nmacdevfsdirents,
    nmacipqs, nmacpipes, nmacprocs;

You can inspect them using a series of sysctls in the security.mac.debug
tree; I'd be interested to see how those values change as you approach the
hang.  If you have DDB compiled into the kernel (which you probably will
want for debugging this problem), I'd be interested in seeing if you can
break into DDB using a console break or serial console break.  If so,
print out the values of those variables and see if any are through the
roof.  Also, you might do a ps in DDB and see if processes are blocked on
anything in particular.

The most likely locking problem is a vnode deadlock; if you can get into
DDB, try "show lockedvnods" to see whether any locks are held, etc.  If
you don't have a serial console set up, you might want to set one up
because it will make copying and pasting all this stuff into e-mail a lot
easier.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Network Associates Laboratories