Bosko Milekic wrote: > > othermark wrote: >>I get a very similar stack track traversing through sossend(), under heavy >>NFS load on a 1GB machine. Note the panic message here, and the >>peculiarity that previous incarnations of -current did not panic under >>similar load. It is highly reproduceable via a 'make installworld' via >>NFS with /usr/src and /usr/obj mounted. The NFS serving machine will >>always panic using vanilla GENERIC: >> >>[root_at_pippin root]$ panic: kmem_malloc(4096): kmem_map too small: 40894464 >>total >> allocated > > Do you have the kern.ipc.nmbclusters boot-time tunable set to 0? > > I just noticed that if this is set to zero then kmem_map will not > be scaled larger to accomodate clusters and mbufs. In this scenario, > what I recommend that you do is increase the vm.kmem_size boot-time > tunable to ~300,000,000 or ~400,000,000. Currently, your kmem_map > is way too small (looks like only ~40M). Be careful not to overdo > it, though, because you might also then have to increase the > available KVA (KVA_PAGES). > > -Bosko It's all a default installation (I try to keep the machines as untweaked as possible). The only thing in loader.conf is to turn on DMA for the hdd. I rebooted with kern.ipc.nmbclusters="0" and vm.kmem_size="300000000" in /boot/loader.conf. Another NFS installworld completed successfully (but appeared to going slower). I'm assuming that GENERIC w/ no adjustments should complete this type of task w/o panicing anyway? Thanks for your help Bosko! Here are the vmstat, netstat numbers that Robert requested earlier in this thread after the successful installworld. Hopefully the high/peak numbers will be useful in analysis (long): [root_at_pippin root]$ vmstat -m Type InUse MemUse HighUse Requests Size(s) acpidev 36 1K 1K 36 16 acpisem 15 1K 1K 15 64 acpitask 0 0K 1K 2 16,32 acpica 945 56K 56K 15810 16,32,64,128,256,512,1024,2048 atkbddev 2 1K 1K 2 32 pfs_fileno 1 20K 20K 1 pfs_nodes 20 3K 3K 20 128 nexusdev 3 1K 1K 3 16 memdesc 1 4K 4K 1 4096 MSDOSFS mount 1 16K 16K 1 UMAHash 2 3K 3K 5 256,512,1024,2048 VM pgdata 2 9K 9K 2 64 DEVFS 110 17K 17K 110 16,32,128,4096 UFS mount 3 7K 7K 3 256,2048,4096 UFS ihash 1 32K 32K 1 UFS dirhash 87 16K 26K 258 16,32,64,128,256,512 dirrem 0 0K 1K 32 32 mkdir 0 0K 1K 6 32 diradd 0 0K 1K 36 32 freefile 0 0K 1K 22 32 freeblks 0 0K 3K 19 256 freefrag 0 0K 1K 8 32 allocdirect 0 0K 2K 70 128 bmsafemap 0 0K 1K 50 32 newblk 1 1K 1K 71 64,256 inodedep 2 33K 39K 119 128 pagedep 1 4K 5K 11 64,4096 p1003.1b 1 1K 1K 1 16 agp 1 1K 1K 1 16 NFS daemon 69 7K 7K 69 64,256,512 NFSV3 srvdesc 0 0K 1K 321570 16,256 NFS srvsock 2 1K 1K 2 128 NFS hash 1 32K 32K 1 ip6_moptions 1 1K 1K 1 16 in6_multi 11 1K 1K 11 16,64 syncache 1 8K 8K 1 hostcache 1 24K 24K 1 in_multi 2 1K 1K 2 32 routetbl 55 8K 8K 88 16,32,64,128,256 USBdev 1 1K 2K 4 128,512 USB 15 2K 2K 15 16,32,64,128,256 lo 1 1K 1K 1 512 clone 4 16K 16K 4 4096 ether_multi 37 2K 2K 37 16,32,64 ifaddr 65 14K 14K 65 16,32,64,256,512,2048 BPF 6 1K 1K 6 64 mount 10 3K 3K 14 16,32,128,512,1024 vnodes 28 7K 7K 159 16,32,64,128,256 cluster_save buffer 0 0K 1K 1 32 vfscache 1 64K 64K 1 BIO buffer 288 576K 740K 3080 2048 pcb 26 5K 5K 100 16,32,64,2048 soname 2 1K 1K 162678 16,32,128 tag 0 0K 1K 5 32 ttys 1013 134K 134K 2378 128,512 shm 1 12K 12K 1 sem 4 7K 7K 4 512,1024,4096 msg 4 25K 25K 4 512,4096 ioctlops 0 0K 1K 14 512,1024 entropy 1024 64K 64K 1024 64 ppbusdev 3 1K 1K 3 128 turnstiles 111 7K 7K 111 64 taskqueue 5 1K 1K 5 64 sleep queues 111 4K 4K 111 32 sbuf 0 0K 13K 190 16,32,64,128,256,512,1024,2048,4096 rman 126 8K 8K 486 16,64 kobj 219 438K 438K 264 2048 eventhandler 33 2K 2K 33 32,128 devstat 4 9K 9K 4 16,4096 bus-sc 45 27K 111K 1323 16,32,64,128,256,512,1024,2048,4096 bus 841 34K 112K 3159 16,32,64,128,1024 SWAP 2 209K 209K 2 64 sysctltmp 0 0K 1K 724 16,32,64,128 sysctloid 1847 56K 56K 1847 16,32,64 sysctl 0 0K 1K 310 16,32,64 uidinfo 3 1K 1K 8 32,128 plimit 16 4K 5K 2974 256 cred 13 2K 3K 4272 128 subproc 78 312K 328K 879 4096 proc 2 1K 1K 2 512 session 24 3K 4K 282 128 pgrp 26 2K 2K 303 64 mtx_pool 1 8K 8K 1 module 329 21K 21K 329 64,128 ip6ndp 8 1K 1K 9 64,128 temp 35 78K 87K 37496 16,32,64,128,256,512,1024,2048,4096 devbuf 166 868K 868K 174 16,32,64,128,256,512,1024,2048,4096 lockf 3 1K 1K 11 64 linker 49 2K 2K 82 16,32,64,256 KTRACE 100 13K 13K 100 128 ithread 51 5K 5K 52 64,128 zombie 0 0K 1K 801 128 proc-args 30 2K 2K 1123 32,64,128 kqueue 0 0K 1K 22 256 kenv 105 6K 6K 106 16,32,64,2048 sigio 1 1K 1K 1 32 file desc 81 24K 25K 900 32,256,2048 dev_t 35 9K 9K 35 256 ACD driver 1 2K 2K 1 2048 AR driver 1 1K 3K 3 64,256,2048 AD driver 1 1K 1K 1 128 ATA DMA 2 1K 1K 2 128 ATA generic 4 2K 2K 4 16,512 ISOFS mount 1 32K 32K 1 isadev 19 2K 2K 19 64 GEOM 49 6K 15K 234 16,32,64,128,256,512,1024,2048,4096 [root_at_pippin root]$ vmstat -z ITEM SIZE LIMIT USED FREE REQUESTS FFS2 dinode: 256, 0, 0, 0, 0 FFS1 dinode: 128, 0, 5851, 39, 17801 FFS inode: 140, 0, 5851, 29, 17801 SWAPMETA: 276, 14980, 0, 0, 0 unpcb: 140, 0, 9, 47, 315 ripcb: 180, 1936, 0, 44, 2 tcpreass: 20, 0, 0, 0, 0 hostcache: 88, 15390, 1, 89, 1 syncache: 104, 15390, 0, 76, 261 tcptw: 56, 426, 0, 0, 0 tcpcb: 372, 1930, 17, 13, 282 inpcb: 180, 1936, 17, 27, 282 udpcb: 180, 1936, 18, 26, 141 socket: 244, 1936, 44, 20, 745 KNOTE: 64, 0, 0, 0, 0 PIPE: 376, 0, 3, 17, 250 NFSNODE: 452, 0, 0, 0, 0 NFSMOUNT: 424, 0, 0, 0, 0 DIRHASH: 1024, 0, 85, 11, 255 NAMEI: 1024, 0, 0, 12, 88425 L VFS Cache: 291, 0, 3, 36, 23 S VFS Cache: 68, 0, 6175, 89, 40269 VNODEPOLL: 60, 0, 0, 0, 0 VNODE: 260, 0, 5870, 10, 5870 ata_request: 180, 0, 0, 44, 18511 g_bio: 136, 0, 0, 116, 72421 MbufClust: 2048, 0, 384, 124, 124710 Mbuf: 256, 0, 160051, 104, 697905 Packet: 256, 0, 159987, 168, 167937 VMSPACE: 236, 0, 30, 21, 853 UPCALL: 44, 0, 0, 0, 0 KSE: 84, 0, 99, 65, 99 KSEGRP: 128, 0, 99, 56, 99 TID: 140, 0, 0, 0, 0 THREAD: 340, 0, 99, 11, 99 PROC: 440, 0, 77, 22, 900 Files: 68, 0, 94, 80, 6712 4096: 4096, 0, 117, 17, 7001 2048: 2048, 0, 525, 63, 3476 1024: 1024, 0, 13, 39, 759 512: 512, 0, 90, 30, 922 256: 256, 0, 211, 29, 165379 128: 128, 0, 1479, 71, 11187 64: 64, 0, 3130, 32, 20854 32: 32, 0, 1043, 187, 5661 16: 16, 0, 1889, 262, 348635 DP fakepg: 72, 0, 0, 0, 0 PV ENTRY: 28, 225680, 7576, 3064, 397167 MAP ENTRY: 60, 0, 568, 224, 33578 KMAP ENTRY: 60, 7656, 23, 109, 20105 MAP: 160, 0, 7, 18, 0 VM OBJECT: 132, 0, 5020, 110, 30065 128 Bucket: 524, 0, 31, 4, 0 64 Bucket: 268, 0, 16, 14, 0 32 Bucket: 140, 0, 23, 5, 0 16 Bucket: 76, 0, 18, 34, 0 UMA Hash: 128, 0, 2, 29, 0 UMA RCntSlab: 82, 0, 254, 28, 0 UMA Slabs: 38, 0, 507, 87, 0 UMA Zones: 328, 0, 59, 7, 0 UMA Kegs: 136, 0, 59, 16, 0 [root_at_pippin root]$ netstat -mb 159963 mbufs in use 256/0 mbuf clusters in use (current/max) 0/2/1456 sfbufs in use (current/peak/max) 40502 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 34 calls to protocol drain routines -- othermark atkin901 at nospam dot yahoo dot com (!wired)?(coffee++):(wired);Received on Fri Jun 11 2004 - 14:27:16 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:56 UTC