Re: libthr and main thread stack size

On Fri, Aug 08, 2014 at 12:32:56PM +0400, Ivan A. Kosarev wrote:
> 
> On 08/08/2014 09:28 AM, Konstantin Belousov wrote:
> > On Thu, Aug 07, 2014 at 04:18:12PM +0400, Ivan A. Kosarev wrote:
> >> Hello,
> >>
> >> According to libthr's thr_init.c (the 9.2 version) init_main_thread()
> >> allocates s.c. "red zone" below the main stack in order to protect other
> >> stacks. The size of the main stack is determined by the
> >> _thr_stack_initial variable that is declared extern though it doesn't
> >> seem it can be changed. The value of the variable is set to 4M on 64-bit
> >> platforms which is obviously not sufficient for the most of real programs.
> >>
> >> Can anyone please confirm that there is no way to increase the stack
> >> size for the main thread and thus any program linked against libthr has
> >> only a few megabytes of stack memory for its main thread--whatever the
> >> system stack size (ulimit -s) is set to?
> > Yes, there is no way to change the main thread stack clamping.
> > Could you provide a reasonable use case for the 4MB stack ?
> 
> Traversing trees with recursive functions or on-stack grammar parsers?
> 
> >
> > Anyway, I somewhat sympathize to the idea to stop clamping the main
> > thread stack, and to not reuse it for other threads stack carving.
> > This also means that non-main threads stack allocator should stop
> > tracking the explicit location for the stacks and rely on vm mmap(2)
> > base selection instead.
> 
> Yes, that would solve the problem.
> 
> > I do not know the motivations why the current scheme of stacks allocation
> > was chosen.  The changes do not look too involved.
In fact, I can resonably explain the current behaviour. The motivation
seems to come from desire to interpret the RLIMIT_STACK as the global
limit to the stack usage by all threads. From this PoV, it becomes clean
why the other thread stacks are carved from the main stack.

Of course, it does not quite work this way, because there is no check
that we did not overflowed from the main stack area.

> 
> Thanks a lot.
> 

Below is the patch which adds environment variable LIBPTHREAD_BIGSTACK_MAIN.
Setting it to any value results in the main thread stack left as is, and
other threads allocate stack below the area of RLIMIT_STACK.  Try it.
I do not want to set this behaviour as default.

diff --git a/lib/libthr/thread/thr_init.c b/lib/libthr/thread/thr_init.c
index 937d83f..9bf0e29 100644
--- a/lib/libthr/thread/thr_init.c
+++ b/lib/libthr/thread/thr_init.c
_at__at_ -37,6 +37,7 _at__at_
 #include <sys/types.h>
 #include <sys/signalvar.h>
 #include <sys/ioctl.h>
+#include <sys/resource.h>
 #include <sys/sysctl.h>
 #include <sys/ttycom.h>
 #include <sys/mman.h>
_at__at_ -441,6 +442,7 _at__at_ init_main_thread(struct pthread *thread)
 static void
 init_private(void)
 {
+	struct rlimit rlim;
 	size_t len;
 	int mib[2];
 	char *env;
_at__at_ -471,6 +473,12 _at__at_ init_private(void)
 		len = sizeof (_usrstack);
 		if (sysctl(mib, 2, &_usrstack, &len, NULL, 0) == -1)
 			PANIC("Cannot get kern.usrstack from sysctl");
+		env = getenv("LIBPTHREAD_BIGSTACK_MAIN");
+		if (env != NULL) {
+			if (getrlimit(RLIMIT_STACK, &rlim) == -1)
+				PANIC("Cannot get stack rlimit");
+			_thr_stack_initial = rlim.rlim_cur;
+		}
 		len = sizeof(_thr_is_smp);
 		sysctlbyname("kern.smp.cpus", &_thr_is_smp, &len, NULL, 0);
 		_thr_is_smp = (_thr_is_smp > 1);
diff --git a/lib/libthr/thread/thr_stack.c b/lib/libthr/thread/thr_stack.c
index 15a9c82..e5d149e 100644
--- a/lib/libthr/thread/thr_stack.c
+++ b/lib/libthr/thread/thr_stack.c
_at__at_ -246,7 +246,10 _at__at_ _thr_stack_alloc(struct pthread_attr *attr)
 		THREAD_LIST_UNLOCK(curthread);
 	}
 	else {
-		/* Allocate a stack from usrstack. */
+		/*
+		 * Allocate a stack from or below usrstack, depending
+		 * on the LIBPTHREAD_BIGSTACK_MAIN env variable.
+		 */
 		if (last_stack == NULL)
 			last_stack = _usrstack - _thr_stack_initial -
 			    _thr_guard_default;
_at__at_ -268,7 +271,7 _at__at_ _thr_stack_alloc(struct pthread_attr *attr)
 
 		/* Map the stack and guard page together, and split guard
 		   page from allocated space: */
-		if ((stackaddr = mmap(stackaddr, stacksize+guardsize,
+		if ((stackaddr = mmap(stackaddr, stacksize + guardsize,
 		     _rtld_get_stack_prot(), MAP_STACK,
 		     -1, 0)) != MAP_FAILED &&
 		    (guardsize == 0 ||