Heap size limit in java

I originally posed to my question to freebsd-java. My issue was that I 
was unable to set a java heap size of more than approx 2000m using java 
(-Xmx2000m), it would fail with the error "COuld not reserve enough 
space for object heap.

Panagiotis Astithas looked into this a bit further and found that it 
was due to the JVM being unable to mmap 3gb or more of memory. His 
explanation is below along with some of the background information.

Any help would be greatly appreciated. I imagine others have hit this 
issue or will shortly. It's somewhat critical to getting large scale 
java applications running on FreeBSD that the JVM be able to access 
enough memory.

Thanks,
Paul


From: Panagiotis Astithas <past_at_noc.ntua.gr>
Date: May 24, 2004 3:57:27 AM EDT
To: Greg Lewis <glewis_at_eyesbeyond.com>
Cc: Paul Dlug <paul_at_aps.org>, freebsd-java_at_freebsd.org
Subject: Re: Heap size limit

Greg Lewis wrote:
> On Fri, Apr 23, 2004 at 09:49:27AM -0400, Paul Dlug wrote:
>> I'm trying to start a JVM with a large heap size and getting an error 
>> message:
>>
>> java -Xms256m -Xmx2024m
>> Error occurred during initialization of VM
>> Could not reserve enough space for object heap
>>
>> I tried a bunch of different sizes and anything under 2024m works 
>> just fine so this appears to be a hard limit. I've tried changing a 
>> few system parameters but can't seem to stumble on the right one. Can 
>> someone help me out with this?
> Obviously I'm a long way behind on freebsd-java, but I'm catching up 
> now...
> My strong suggestion is that you're hitting the maximum amount of 
> memory
> that your user can mmap.  I'm not exactly sure what setting dictate 
> that,
> but its almost certainly what you're running into.

Definitely:
ktrace java -Xms256m -Xmx2024m

[...]
  15034 java     CALL  mmap(0,0x82800000,0,0x1042,0xffffffff,0,0,0)
  15034 java     RET   mmap -1 errno 22 Invalid argument
  15034 java     CALL  write(0x1,0xbfbfd670,0x2b)
  15034 java     GIO   fd 1 wrote 43 bytes
        "Error occurred during initialization of VM
        "
  15034 java     RET   write 43/0x2b
  15034 java     CALL  write(0x1,0xbfbfd670,0x2e)
  15034 java     GIO   fd 1 wrote 46 bytes
        "Could not reserve enough space for object heap"
[...]


% grep 22 /usr/include/errno.h
#define EINVAL          22              /* Invalid argument */

%man mmap
[...]

      [EINVAL]    MAP_FIXED was specified and the addr argument was not
                  page aligned, or part of the desired address space
                  resides out of the valid address space for a user
                  process.
[...]


Now try changing vm.max_proc_mmap to an insanely large value:

# sysctl vm.max_proc_mmap=10000000000000
vm.max_proc_mmap: 26960 -> 2147483647

We see the requested size gets truncated to 2 GB.
This still doesn't help us, though:

# /usr/local/jdk1.4.2/bin/java -Xms256m -Xmx2024m
Error occurred during initialization of VM
Could not reserve enough space for object heap


The weird thing is that in a GENERIC kernel, the kernel/userland 
address space split is 1 GB / 3 GB. As it can bee sen from the 
following excerpt from /sys/i386/conf/NOTES:

#
# Change the size of the kernel virtual address space.  Due to
# constraints in loader(8) on i386, this must be a multiple of 4.
# 256 = 1 GB of kernel address space.  Increasing this also causes
# a reduction of the address space in user processes.  512 splits
# the 4GB cpu address space in half (2GB user, 2GB kernel).
#
options         KVA_PAGES=260


Why you can't mmap up to 3 GB of memory, I don't know. Perhaps we 
should take it to -current or -hackers.

Cheers,
-- 
Panagiotis Astithas
Electrical & Computer Engineer, PhD
Network Management Center
National Technical University of Athens, Greece