Re: HEADS UP: sparc64 backend for llvm/clang imported

On Wed, Aug 26, 2015 at 11:14:51PM +0200, Marius Strobl wrote:
> On Wed, Aug 19, 2015 at 04:19:03PM -0400, Kurt Lidl wrote:
> > > Dimitry Andric wrote this message on Fri, Feb 28, 2014 at 20:22 +0100:
> > >> In r262613 I have merged the clang-sparc64 branch back to head.  This
> > >> imports an updated sparc64 backend for llvm and clang, allowing clang to
> > >> bootstrap itself on sparc64, and to completely build world.  To be able
> > >> to build the GENERIC kernel, there is still one patch to be finalized,
> > >> see below.
> > >>
> > >> If you have any sparc64 hardware, and are not afraid to encounter rough
> > >> edges, please try out building and running your system with clang.  To
> > >> do so, update to at least r262613, and enable the following options in
> > >> e.g. src.conf, or in your build environment:
> > >>
> > >> WITH_CLANG=y
> > >> WITH_CLANG_IS_CC=y
> > >> WITH_LIBCPLUSPLUS=y  (optional)
> > >>
> > >> Alternatively, if you would rather keep gcc as /usr/bin/cc for the
> > >> moment, build world using just WITH_CLANG, enabling clang to be built
> > >> (by gcc) and installed.  After installworld, you can then set CC=clang,
> > >> CXX=clang++ and CPP=clang-cpp for building another world.
> > >>
> > >> For building the sparc64 kernel, there is one open issue left, which is
> > >> that sys/sparc64/include/pcpu.h uses global register variables, and this
> > >> is not supported by clang.  A preliminary patch for this is attached,
> > >> but it may or may not blow up your system, please beware!
> > >>
> > >> The patch changes the pcpu and curpcb global register variables into
> > >> inline functions, similar to what is done on other architectures.
> > >> However, the current approach is not optimal, and the emitted code is
> > >> slightly different from what gcc outputs.  Any improvements to this
> > >> patch are greatly appreciated!
> > >>
> > >> Last but not least, thanks go out to Roman Divacky for his work with
> > >> llvm/clang upstream in getting the sparc64 backend into shape.
> > >
> > > Ok, I have a new pcpu patch to try.  I have only compile tested it.
> > >
> > > It is available here:
> > > https://www.funkthat.com/~jmg/sparc64.pcpu.patch
> > >
> > > I've also attached it.
> > >
> > > Craig, do you mind testing it?
> > >
> > > This patch also removes curpcb as it appears to not be used by any
> > > sparc64 C code.  A GENERIC kernel compiles fine, and fxr only turns up
> > > curpcb used in machdep code, and no references to it under sparc64.
> > >
> > > This is not a proper solution in that
> > > it can mean counters/stats can be copied/moved to other cpus overwriting
> > > the previous values if a race happens...  We use
> > > PCPU_SET(mem, PCPU_GET(mem) + val) for PCPU_ADD, not great, but it's
> > > no worse than what we were previously using..
> > >
> > > Until we get a proper fix which involves mapping all the cpu's PCPU
> > > data on all CPUs, this will have to sufice..
> > >
> > > This patch is based upon, I believe, a patch from Marius and possibly
> > > modified by rdivacky.
> > >
> > > Thanks for testing..
> > 
> > The above message was posted a while ago, and I decided that I would
> > give the patch a test run on a spare sparc that I have, now that the
> > instability problem with multiprocessor sparc64 machines has been
> > resolved.
> > 
> > So, I have an up-to-date stable/10 V240 (2x1.5Ghz cpus, 8GB of memory),
> > running a completely stock r286861.  That all seems to work just fine.
> > 
> > I applied the patch referenced in the email:
> > 
> > https://www.funkthat.com/~jmg/sparc64.pcpu.patch
> > 
> > (it applied cleanly), and then rebuilt the kernel on the machine,
> > using the stock gcc 4.2.1 compiler.
> > 
> > When rebooting with that kernel, the machine panics pretty early
> > in the boot:
> > 
> > FreeBSD 10.2-STABLE #3 r286861M: Wed Aug 19 14:28:45 EDT 2015
> >      lidl_at_spork.pix.net:/usr/obj/usr/src/sys/GENERIC sparc64
> > gcc version 4.2.1 20070831 patched [FreeBSD]
> > real memory  = 8589934592 (8192 MB)
> > avail memory = 8379719680 (7991 MB)
> > cpu0: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU)
> > cpu1: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU)
> > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> > random device not loaded; using insecure entropy
> > panic: trap: illegal instruction (kernel)
> > cpuid = 0
> > KDB: stack backtrace:
> > #0 0xc05750e0 at panic+0x20
> > #1 0xc08db9f8 at trap+0x558
> > Uptime: 1s
> > Automatic reboot in 15 seconds - press a key on the console to abort
> > Rebooting...
> > timeout shutting down CPUs.
> > 
> > So, the patch to get rid of the pcpu usage (as a prereq to poking
> > at the clang compiler issues) does not work properly.
> > 
> 
> As I pointed out when that patch was posted, the approach taken by
> it assumes a CPU can access foreign PCPU data, which currently isn't
> true on sparc64. So the patch is at least incomplete but also may
> have further issues.
> 
> Such a patch is no longer a prerequisite for compiling a sparc64
> kernel with clang, though, as clang meanwhile has been told to
> grok at least the global registers used by the PCPU code.
> 
> Besides some default options like the choice of code model not
> being appropriate for FreeBSD, clang-compiled loader and kernel
> don't work due to two major problems present in clang up to at
> least version 3.6.0: a) it uses a different stack layout than
> GCC so any unwinding code fails and b) it produces broken code
> for recursive function calls. YMMV.

Is there a bug reported upstream about the recursive function calls?

Roman