Re: ZFS kmem_map too small.

On Mon, Oct 08, 2007 at 03:30:41PM +0200, Dag-Erling Sm??rgrav wrote:
> Pawel Jakub Dawidek <pjd_at_FreeBSD.org> writes:
> > I'm not sure if it's not too late to ask re_at_ about increasing the
> > default kmem size at least on amd64. ~300MB we have there is silly
> > small.
> 
> Speaking of which, I tried setting vm.kmem_size to 2G on a C2D system
> with 4 GB RAM, but it simply panics:
> 
> OK set vm.kmem_size=2G
> OK boot -s
> GDB: no debug ports present
> KDB: debugger backends: ddb
> KDB: current backend: ddb
> Copyright (c) 1992-2007 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>         The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 7.0-CURRENT #8: Tue Sep 25 13:31:41 CEST 2007
>     des_at_ds4.des.no:/usr/obj/usr/src/sys/ds4
> kmem_suballoc: bad status return of 3.
> panic: kmem_suballoc
> cpuid = 0
> KDB: enter: panic
> [thread pid 0 tid 0 ]
> Stopped at      kdb_enter+0x31: popq    %rbp
> db> where
> Tracing pid 0 tid 0 td 0xffffffff805af4a0
> kdb_enter() at kdb_enter+0x31
> panic() at panic+0x166
> kmem_suballoc() at kmem_suballoc+0xc3
> kmeminit() at kmeminit+0x16e
> mi_startup() at mi_startup+0x59
> btext() at btext+0x2c
> db> reset
> 
> with vm.kmem_size unset and vm.kmem_size_max=2G, it boots fine:
> 
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Core(TM)2 CPU          6600  _at_ 2.40GHz (2400.01-MHz K8-class CPU)
>   Origin = "GenuineIntel"  Id = 0x6f6  Stepping = 6
>   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>   Features2=0xe3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
>   AMD Features=0x20100800<SYSCALL,NX,LM>
>   AMD Features2=0x1<LAHF>
>   Cores per package: 2
> usable memory = 4286398464 (4087 MB)
> avail memory  = 4130979840 (3939 MB)
> ACPI APIC Table: <GBT    GBTUACPI>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  1
> ioapic0: Changing APIC ID to 2
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> [...]

For i386 one has to set, eg. 'options KVA_PAGES=512' to the kernel
config to be able to define kmem larger than ~700MB. I guess you're
running amd64, maybe there is similar requirement?

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd_at_FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!