5.1R + mdconfig -t malloc + cp = Panic!?

From: Jay Kuri <jay_at_oneway.com>
Date: Sat, 14 Jun 2003 12:58:28 -0400 (EDT)
Good Morning,

I believe I have stumbled onto a bug in 5.1R.  

Here is the problem.  If you have a 5.1-R system with, say, 256M of ram
and you mdconfig one or several malloc md-devices totalling around 100M,
if you attempt to use them to capacity, the machine will panic.  The
message given is:

 panic: kmem_malloc(4096): kmem_map too small: 108347392 total allocated

I have taken some time to track down how to repeat the problem at will...
so perhaps someone can figure it out.  The problem does not occur if you
have more memory, I tried it on the exact same configuration, only with 4G
of ram, and there is no panic. I have also tried it on several machines,
with the same results.  I would guess that it has something to do with
caching/buffering of the copied files, but that's just a guess.

I have attached a shell script that will trigger the error.  I've run this
several times, usually in single-user mode, and without fail on a 256M
machine it will panic.  set 'cache_dirs' to any set of directories that
will total 100M or more.  On my specific config I have a default 5.1R
install with a minimal install+catman, the dirs listed in the script total
around 110M.

anyone have any ideas?  The script that got me started on this whole thing
worked fine on 5.0-Current (Apr 30), but panic'd the machine as of
5.1R.  (it's a different script than the one below, but I removed the
non-essential parts of the script to make it less confusing)  I should
also mention that the kernel is identical to 'GENERIC' except that 
'options POLLING' was included.

Below is the script, and a log of what occurs on a 4G machine, then a 256M
machine...

Any help would be appreciated,

Jay Kuri

- shell script -

#!/bin/sh

## quick script to show the mdconfig -t malloc + fill = panic bug on 5.1-R
PATH=/bin:/sbin:/usr/bin:/usr/sbin

# on a machine with 256M of ram, this is enough to crash it. 112M roughly on
# my machine.  Use any combination of directories that will push the limit
# up over 100M 
cache_dirs="/bin /sbin /usr/bin /usr/sbin /usr/lib"

# say we have 100M less than we actually have, to make sure we don't
# try to use more memory than we have 
space_in_k=`sysctl hw.physmem|awk '{print int($2 / 1024) - 100000}'`

echo Ram available to us: $space_in_k
totaldirsize=0

type="malloc"
## This gives us a writable directory so that we can run this off of the 5.1 
## install or live cd.  
mddev=`/sbin/mdconfig -a -t malloc -s 1M`
/sbin/newfs /dev/${mddev}
/sbin/mount /dev/${mddev} /mnt
mkdir /mnt/copy
/bin/cp /bin/cp /mnt/

for dir in ${cache_dirs}
do
      dirsize=`/usr/bin/du -sk ${dir}|/usr/bin/awk '{print int(1024 + ($1 * 1.20))}'`
      totaldirsize=`/bin/expr ${totaldirsize} + ${dirsize}`
      if [ ${totaldirsize} -le ${space_in_k} ]; then
	   dirs_mirrored=`echo ${dirs_mirrored} $dir`
	   mddevice=`/sbin/mdconfig -a -t ${type} -s ${dirsize}k`
	   /sbin/newfs -i 1024 -U /dev/${mddevice} >/dev/null
	   /sbin/tunefs -m 2 /dev/${mddevice}
	   /sbin/mount /dev/${mddevice} /mnt/copy >/dev/null
	   /mnt/cp -Rp ${dir}/ /mnt/copy
	   /sbin/umount /mnt/copy >/dev/null
	   /bin/mkdir -p /mnt${dir}
	   /sbin/mount /dev/${mddevice} /mnt${dir} >/dev/null
      fi
done

- fin - 

- 4G log -
# df -k         
Filesystem 1K-blocks   Used  Avail Capacity  Mounted on
/dev/acd0     345120 345120      0   100%    /
devfs              1      1      0   100%    /dev
/dev/da0s1    253888  84416 169472    33%    /opt
# sh -x test.sh
+ PATH=/bin:/sbin:/usr/bin:/usr/sbin
+ cache_dirs=/bin /sbin /usr/bin /usr/sbin /usr/lib
+ sysctl+ awk {print int($2 / 1024) - 100000}
 hw.physmem
+ space_in_k=3928684
+ echo Ram available to us: 3928684
Ram available to us: 3928684
+ totaldirsize=0
+ type=malloc
+ /sbin/mdconfig -a -t malloc -s 1M
+ mddev=md0
+ /sbin/newfs /dev/md0
/dev/md0: 1.0MB (2048 sectors) block size 16384, fragment size 2048
        using 2 cylinder groups of 0.50MB, 32 blks, 64 inodes.
super-block backups (for fsck -b #) at:
 160, 1184
+ /sbin/mount /dev/md0 /mnt
+ mkdir /mnt/copy
+ /bin/cp /bin/cp /mnt/
+ /usr/bin/du+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
 -sk /bin
+ dirsize=12050
+ /bin/expr 0 + 12050
+ totaldirsize=12050
+ [ 12050 -le 3928684 ]
+ echo /bin
+ dirs_mirrored=/bin
+ /sbin/mdconfig -a -t malloc -s 12050k
+ mddevice=md1
+ /sbin/newfs -i 1024 -U /dev/md1
+ /sbin/tunefs -m 2 /dev/md1
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md1 /mnt/copy
+ /mnt/cp -Rp /bin/ /mnt/copy
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/bin
+ /sbin/mount /dev/md1 /mnt/bin
+ /usr/bin/du -sk /sbin
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=33913
+ /bin/expr 12050 + 33913
+ totaldirsize=45963
+ [ 45963 -le 3928684 ]
+ echo /bin /sbin
+ dirs_mirrored=/bin /sbin
+ /sbin/mdconfig -a -t malloc -s 33913k
+ mddevice=md2
+ /sbin/newfs -i 1024 -U /dev/md2
+ /sbin/tunefs -m 2 /dev/md2
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md2 /mnt/copy
+ /mnt/cp -Rp /sbin/ /mnt/copy
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/sbin
+ /sbin/mount /dev/md2 /mnt/sbin
+ /usr/bin/du -sk /usr/bin
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=34375
+ /bin/expr 45963 + 34375
+ totaldirsize=80338
+ [ 80338 -le 3928684 ]
+ echo /bin /sbin /usr/bin
+ dirs_mirrored=/bin /sbin /usr/bin
+ /sbin/mdconfig -a -t malloc -s 34375k
+ mddevice=md3
+ /sbin/newfs -i 1024 -U /dev/md3
+ /sbin/tunefs -m 2 /dev/md3
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md3 /mnt/copy
+ /mnt/cp -Rp /usr/bin/ /mnt/copy
/mnt/copy: optimization changed from TIME to SPACE
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/usr/bin
+ /sbin/mount /dev/md3 /mnt/usr/bin
+ /usr/bin/du -sk /usr/sbin
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=10420
+ /bin/expr 80338 + 10420
+ totaldirsize=90758
+ [ 90758 -le 3928684 ]
+ echo /bin /sbin /usr/bin /usr/sbin
+ dirs_mirrored=/bin /sbin /usr/bin /usr/sbin
+ /sbin/mdconfig -a -t malloc -s 10420k
+ mddevice=md4
+ /sbin/newfs -i 1024 -U /dev/md4
+ /sbin/tunefs -m 2 /dev/md4
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md4 /mnt/copy
+ /mnt/cp -Rp /usr/sbin/ /mnt/copy
/mnt/copy: optimization changed from TIME to SPACE
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/usr/sbin
+ /sbin/mount /dev/md4 /mnt/usr/sbin
+ /usr/bin/du -sk /usr/lib
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=49560
+ /bin/expr 90758 + 49560
+ totaldirsize=140318
+ [ 140318 -le 3928684 ]
+ echo /bin /sbin /usr/bin /usr/sbin /usr/lib
+ dirs_mirrored=/bin /sbin /usr/bin /usr/sbin /usr/lib
+ /sbin/mdconfig -a -t malloc -s 49560k
+ mddevice=md5
+ /sbin/newfs -i 1024 -U /dev/md5
+ /sbin/tunefs -m 2 /dev/md5
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md5 /mnt/copy
+ /mnt/cp -Rp /usr/lib/ /mnt/copy
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/usr/lib
+ /sbin/mount /dev/md5 /mnt/usr/lib
# df -k
Filesystem 1K-blocks   Used  Avail Capacity  Mounted on
/dev/acd0     345120 345120      0   100%    /
devfs              1      1      0   100%    /dev
/dev/da0s1    253888  84416 169472    33%    /opt
/dev/md0         846    110    670    14%    /mnt
/dev/md1       10304   9528    570    94%    /mnt/bin
/dev/md2       29414  28644    182    99%    /mnt/sbin
/dev/md3       29812  29088    128   100%    /mnt/usr/bin
/dev/md4        8866   8270    420    95%    /mnt/usr/sbin
/dev/md5       43142  43074   -794   102%    /mnt/usr/lib
# top
last pid:   136;  load averages:  0.01,  0.08,  0.06  up 0+00:05:06    15:40:18
2 processes:   1 running, 1 sleeping

Mem: 56M Active, 17M Inact, 168M Wired, 42M Cache, 69M Buf, 3608M Free
Swap: 


  PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU    CPU COMMAND
   44 root       8    0   896K   444K wait     0:00  0.00%  0.00% sh
  136 root      96    0  2136K  1300K RUN      0:00  0.00%  0.00% top


- fin -
- test log 256M -
# top
last pid:    54;  load averages:  0.35,  0.24,  0.10  up 0+00:02:05    16:17:12
2 processes:   1 running, 1 sleeping

Mem: 1184K Active, 268K Inact, 8328K Wired, 3664K Buf, 235M Free
Swap: 


  PID USERNAME PRI NICE   SIZE    RES STATE    TIME   WCPU    CPU COMMAND
   54 root      96    0  2136K   780K RUN      0:00  0.00%  0.00% top
   48 root       8    0   900K   320K wait     0:00  0.00%  0.00% sh

# df -k  
Filesystem 1K-blocks   Used  Avail Capacity  Mounted on
/dev/acd0     345120 345120      0   100%    /
devfs              1      1      0   100%    /dev
/dev/da0s1    253888  84416 169472    33%    /opt
# sh -x test.sh
+ PATH=/bin:/sbin:/usr/bin:/usr/sbin
+ cache_dirs=/bin /sbin /usr/bin /usr/sbin /usr/lib
+ sysctl+ awk {print int($2 / 1024) - 100000}
 hw.physmem
+ space_in_k=154980
+ echo Ram available to us: 154980
Ram available to us: 154980
+ totaldirsize=0
+ type=malloc
+ /sbin/mdconfig -a -t malloc -s 1M
+ mddev=md0
+ /sbin/newfs /dev/md0
/dev/md0: 1.0MB (2048 sectors) block size 16384, fragment size 2048
        using 2 cylinder groups of 0.50MB, 32 blks, 64 inodes.
super-block backups (for fsck -b #) at:
 160, 1184
+ /sbin/mount /dev/md0 /mnt
+ mkdir /mnt/copy
+ /bin/cp /bin/cp /mnt/
+ /usr/bin/du+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
 -sk /bin
+ dirsize=12050
+ /bin/expr 0 + 12050
+ totaldirsize=12050
+ [ 12050 -le 154980 ]
+ echo /bin
+ dirs_mirrored=/bin
+ /sbin/mdconfig -a -t malloc -s 12050k
+ mddevice=md1
+ /sbin/newfs -i 1024 -U /dev/md1
+ /sbin/tunefs -m 2 /dev/md1
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md1 /mnt/copy
+ /mnt/cp -Rp /bin/ /mnt/copy
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/bin
+ /sbin/mount /dev/md1 /mnt/bin
+ /usr/bin/du -sk /sbin
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=33913
+ /bin/expr 12050 + 33913
+ totaldirsize=45963
+ [ 45963 -le 154980 ]
+ echo /bin /sbin
+ dirs_mirrored=/bin /sbin
+ /sbin/mdconfig -a -t malloc -s 33913k
+ mddevice=md2
+ /sbin/newfs -i 1024 -U /dev/md2
+ /sbin/tunefs -m 2 /dev/md2
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md2 /mnt/copy
+ /mnt/cp -Rp /sbin/ /mnt/copy
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/sbin
+ /sbin/mount /dev/md2 /mnt/sbin
+ /usr/bin/du -sk /usr/bin
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=34375
+ /bin/expr 45963 + 34375
+ totaldirsize=80338
+ [ 80338 -le 154980 ]
+ echo /bin /sbin /usr/bin
+ dirs_mirrored=/bin /sbin /usr/bin
+ /sbin/mdconfig -a -t malloc -s 34375k
+ mddevice=md3
+ /sbin/newfs -i 1024 -U /dev/md3
+ /sbin/tunefs -m 2 /dev/md3
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md3 /mnt/copy
+ /mnt/cp -Rp /usr/bin/ /mnt/copy
/mnt/copy: optimization changed from TIME to SPACE
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/usr/bin
+ /sbin/mount /dev/md3 /mnt/usr/bin
+ /usr/bin/du -sk /usr/sbin
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=10420
+ /bin/expr 80338 + 10420
+ totaldirsize=90758
+ [ 90758 -le 154980 ]
+ echo /bin /sbin /usr/bin /usr/sbin
+ dirs_mirrored=/bin /sbin /usr/bin /usr/sbin
+ /sbin/mdconfig -a -t malloc -s 10420k
+ mddevice=md4
+ /sbin/newfs -i 1024 -U /dev/md4
+ /sbin/tunefs -m 2 /dev/md4
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md4 /mnt/copy
+ /mnt/cp -Rp /usr/sbin/ /mnt/copy
/mnt/copy: optimization changed from TIME to SPACE
+ /sbin/umount /mnt/copy
+ /bin/mkdir -p /mnt/usr/sbin
+ /sbin/mount /dev/md4 /mnt/usr/sbin
+ /usr/bin/du -sk /usr/lib
+ /usr/bin/awk {print int(1024 + ($1 * 1.20))}
+ dirsize=49560
+ /bin/expr 90758 + 49560
+ totaldirsize=140318
+ [ 140318 -le 154980 ]
+ echo /bin /sbin /usr/bin /usr/sbin /usr/lib
+ dirs_mirrored=/bin /sbin /usr/bin /usr/sbin /usr/lib
+ /sbin/mdconfig -a -t malloc -s 49560k
+ mddevice=md5
+ /sbin/newfs -i 1024 -U /dev/md5
+ /sbin/tunefs -m 2 /dev/md5
tunefs: minimum percentage of free space changes from 8% to 2%
tunefs: should optimize for space with minfree < 8%
+ /sbin/mount /dev/md5 /mnt/copy
+ /mnt/cp -Rp /usr/lib/ /mnt/copy
/mnt/copy: optimization changed from TIME to SPACE
panic: kmem_malloc(4096): kmem_map too small: 108347392 total allocated

syncing disks, buffers remaining... 

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x14
fault code              = supervisor write, page not present
instruction pointer     = 0x8:0xc02d1ad9
stack pointer           = 0x10:0xd24fb544
frame pointer           = 0x10:0xd24fb578
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 137 (cp)
trap number             = 12
panic: page fault
Uptime: 3m12s
Terminate ACPI
Automatic reboot in 15 seconds - press a key on the console to abort

- fin -

- - - - - - - - - - - - - - - - - - - - - - - - - - - 
Nothing fails like success because you do not learn 
anything from it. The only thing we ever learn from 
is failure. Success only confirms our superstitions.

        Jay Kuri	 jay_at_oneway.com
Received on Sat Jun 14 2003 - 07:58:33 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:11 UTC