Re: ARM pcpu changes, was Re: [patch] i386 pmap sysmaps_pcpu[] atomic access

On Fri, Mar 01, 2013 at 02:20:16PM +0100, Svatopluk Kraus wrote:
> On Tue, Feb 26, 2013 at 1:39 PM, Konstantin Belousov
> <kostikbel_at_gmail.com> wrote:
> > On Tue, Feb 26, 2013 at 09:06:51PM +1300, Andrew Turner wrote:
> >> Does anyone know if it is only curthread that needs to be atomic? If so
> >> this should work. Reading the cpuid from the system currently is a
> >> single instruction, however it appears the code will need to be reworked
> >> for use with multiple levels of affinity, for example when there
> >> are clusters of cores the current code will number two cores on
> >> different clusters with the same id.
> >>
> >> I don't think we need to use ldrex/strex to read/write the values, my
> >> understanding is the rest should be safe to access without atomic
> >> functions.
> >
> > I read about ARM architecture some time ago. What I am saying below is not
> > a proposal, but rather a way for me to learn more about the architecture.
> >
> > From my reading of the docs, ARM has the shadow registers, in particular,
> > the r13 (stack pointer) is shadowed for all privileged modes. Is the shadow
> > used by our kernel, is it correctly restored on the context switch ?
> >
> > If yes, you can easily recover the pcb address from the current sp,
> > without accessing any coprocessor registers, and even without any
> > assignment of the global register for curthread (which needs to be
> > done at the kernel entry). Just align the stack
> > start on the 2*PAGE_SIZE boundary (like it is already done for MIPS),
> > and do the mask operation on the sp value to get the end of pcb.
> > This is atomic and context-switch safe.
> >
> > pcb us already per-thread, and can store the thread pointer.
> > More, you can store the curcpu or cpuid pointer into pcb too,
> > and update it on the context switch.
> >
> > amd64 has an architecturally-defined special register (k)gsbase, which
> > effectively must be correctly initialized at each kernel entry, and
> > restored on the return to usermode. Since the initialization on entry
> > and restoration on exit is not atomic wrt the entry/exit itself, amd64
> > periodically gets a bugs which cause kernel running with user gsbase.
> > This is the fatal bug, destroying the kernel structures and allowing the
> > CPU privilege mode switch for normal user.
> >
> > Using the shadow registers for this purpose eliminate the whole source
> > of the bugs.
> 
> Well, IMHO, the both kernel and user thread stacks must correctly be
> set for current threads, otherwise it doesn't work. So, the idea to
> save things on a thread stack and update them on a context switch will
> work in general. However, the stack must be aligned to its size. And
> this is the price. I have no idea how this kernel stack alignment can
> impact kernel virtual space fragmentation.
> OTOH, as you say, this kernel stack storage can be used for many things.
> 

Hi,

FWIW, I have a proof-of-ooncept patch that does just that, and it seems to 
work well.

Regards,

Olivier