Re: kmem_map too small, revisited (5.2.1-REL-p1)

* Sven Willenberger, 2004-03-16 :

> The Issue: Ever since having migrated from 4.x to 5.x I have been having
> an issue (on 2 different hardware setups) with spontaneous reboots or
> system hangs, usually identified by the kmem_map too small message. The

I just got a similar reboot on a 5.2.1-REL-p1 box (ie with the cred leak
fix and the SA 04:04 TCP reassembly queue fix, but with rev. 1.217.2.2
of tcp_input.c). Could that explain the problem?

panic: kmem_malloc(16384): kmem_map too small: 275247104 total allocated
panic: from debugger
Uptime: 61d14h24m27s
Dumping 2047 MB

Here is an excerpt of the backtrace:

#9  0xc06c8b88 in calltrap () at {standard input}:94
#10 0xc0568a55 in panic (
    fmt=0xc07324a1 "kmem_malloc(%ld): kmem_map too small: %ld total
allocated")
    at /usr/src/RELENG_5_2/sys/kern/kern_shutdown.c:534
#11 0xc0691370 in kmem_malloc (map=0xc10310a0, size=16384, flags=1026)
    at /usr/src/RELENG_5_2/sys/vm/vm_kern.c:342
#12 0xc06a2777 in page_alloc (zone=0x0, bytes=0, pflag=0x0, wait=0)
    at /usr/src/RELENG_5_2/sys/vm/uma_core.c:842
#13 0xc06a4302 in uma_large_malloc (size=16384, wait=1026)
    at /usr/src/RELENG_5_2/sys/vm/uma_core.c:2024
#14 0xc055d52b in malloc (size=16384, type=0xc0769100, flags=1026)
    at /usr/src/RELENG_5_2/sys/kern/kern_malloc.c:253
#15 0xc0671f55 in softdep_disk_io_initiation (bp=0xd4a3ff10)
    at /usr/src/RELENG_5_2/sys/ufs/ffs/ffs_softdep.c:3493
#16 0xc0523bd7 in spec_xstrategy (vp=0xc87db618, bp=0xd4a3ff10)
    at /usr/src/RELENG_5_2/sys/sys/buf.h:413
#17 0xc0523d72 in spec_specstrategy (ap=0xe66b1b68)
    at /usr/src/RELENG_5_2/sys/fs/specfs/spec_vnops.c:534
#18 0xc0522eb8 in spec_vnoperate (ap=0x0)
    at /usr/src/RELENG_5_2/sys/fs/specfs/spec_vnops.c:122
#19 0xc06874fc in ufs_strategy (ap=0x0) at vnode_if.h:1141
#20 0xc0688268 in ufs_vnoperate (ap=0x0)
    at /usr/src/RELENG_5_2/sys/ufs/ufs/ufs_vnops.c:2793
#21 0xc05ae5bd in bwrite (bp=0xd4a3ff10) at vnode_if.h:1116
#22 0xc05b03c2 in vfs_bio_awrite (bp=0xd4a3ff10)
    at /usr/src/RELENG_5_2/sys/kern/vfs_bio.c:1715
#23 0xc0679b8c in ffs_fsync (ap=0xe66b1c94)
    at /usr/src/RELENG_5_2/sys/ufs/ffs/ffs_vnops.c:268
#24 0xc0678d04 in ffs_sync (mp=0xc82b0000, waitfor=2, cred=0xd40a4600, 
---Type <return> to continue, or q <return> to quit---
    td=0xd39cd500) at vnode_if.h:627
#25 0xc05c419e in sync (td=0xd39cd500, uap=0xe66b1d14)
    at /usr/src/RELENG_5_2/sys/kern/vfs_syscalls.c:141
#26 0xc06d7d80 in syscall (frame=
      {tf_fs = -1078001617, tf_es = -1078001617, tf_ds = -1078001617,
tf_edi = 0, tf_esi = 136427152, tf_ebp = -1077943608, tf_isp =
-429187724, tf_ebx = 0, tf_edx = 0, tf_ecx = 805742720, tf_eax = 36,
tf_trapno = 12, tf_err = 2, tf_eip = 675618159, tf_cs = 31, tf_eflags =
531, tf_esp = -1077943620, tf_ss = 47})
    at /usr/src/RELENG_5_2/sys/i386/i386/trap.c:1010
#27 0xc06c8bdd in Xint0x80_syscall () at {standard input}:136
---Can't read userspace from dump, or kernel process---

By walking the kmemstatistics data in the crashdump, I was able to
reconstruct kmem usage information (only the biggest figures are listed
here):

Type		MemUse
devbuf		1081216
vfscache	1048576
inodedep	998272
lockf		798080
subproc		764928
SWAP		561216
kobj		239616
ttys		213120
temp		164720
linker		162032
mbufmgr		142608
diradd		118176
acpica		98016
allocdirect	93952
kqueue		83200
pagedep		81408
BPF		74688
entropy		65536
pfs_fileno	40960
mount		34752

I am keeping the crash dump at hand, please let me know if there is any
other piece of data I can provide to help identify the origin of the
problem.

Thanks!
Thomas.

--		
		Thomas.Quinot_at_Cuivre.FR.EU.ORG