Re: [head tinderbox] failure on mips/mips

In message: <BE046876-F8EF-4269-9CD6-743483EC1869_at_mac.com>
            Marcel Moolenaar <xcllnt_at_mac.com> writes:
: 
: On Feb 17, 2009, at 10:58 PM, M. Warner Losh wrote:
: 
: > In message: <B23797BE-91FB-4AE1-8370-E77D66ED05B6_at_mac.com>
: >            Marcel Moolenaar <xcllnt_at_mac.com> writes:
: > :
: > : On Feb 17, 2009, at 10:42 PM, M. Warner Losh wrote:
: > :
: > : > : A safer approach is to mark ifi_epoch as packed or put  
: > differently,
: > : > : define time_t as a 64-bit integral with 32-bit alignment. This  
: > can
: > : > : avoid a lot of unexpected internal padding as well (e.g. struct
: > : > : timeval).
: > : >
: > : > Marking it as packed won't help.  If the elements aren't properly
: > : > aligned, gcc won't access multi-word entities properly.  It might
: > : > eliminate the warning, but it will break at runtime.
: > :
: > : But GCC will use a pair of 32-bit loads and/or stores to
: > : access the 64-bit integral in that case. There should be
: > : no runtime breakage. You only do this for n32 of course.
: >
: > Why only n32?  Registers are still 64-bit in n32.
: 
: I think that's the problem. With registers still 64-bit, MIPS
: n32 isn't really behaving like a 32-bit machine in the case of
: 64-bit accesses. It's that aspect you want to tweak. So, if
: you give all 64-bit integrals an alignment of 4 bytes, then
: GCC will use a pair of 32-bit loads and stores (just like,
: say, powerpc) and you don't run into the alignment problems
: where all of a sudden a data structure gets 8-byte alignment,
: triggers warnings, and we try to correct it with kluges.

MIPS n32 is a 64-bit ABI in many ways, but 32-bit in others.  Its
data-types are 32-bit, but the compiler is free to leverage the
underlying 64-bit machine to implement it.  So for 64-bit quantities,
is uses the ld/sd op codes.

: For MIPS n64 things are like any other LP64 architecture, so
: you don't have to tweak anything.

I think so.

: In other words: by tweaking the alignment of 64-bit types in
: n32, you prohibit GCC from using the 64-bit capabilities of
: the processor and MIPS isn't so weird anymore.

I think so, but there's a twist.

On MIPS, one kernel supports multiple ABIs at the same time.  I'm not
entirely sure how the routing interface would cope with this sort of
thing because the size of u_long changes.  I need to do some more
research to see what's going on there...

: NOTE: On ARM, GCC aligns structures to a 4-byte boundary by
: default. This has caused us problems and instead of fixing
: the default behaviour of the compiler, we slammed __packed
: onto structures. If we had changed the default behaviour of
: the compiler, then all structures would be naturally aligned
: and we would be able to use the half-word memory accesses
: that newer ARM processors have. No, we __packed the lot and
: created a big performance bottleneck because now we can only
: use byte-wise memory accesses.
: What was done for performance (default alignment of 4-bytes
: for structures), was turned into a huge pessimisation by us
: compensating with __packed. We have more optimal code if
: the compiler aligns structures on their natural boundary!

The reason we have the compiler doing this on ARM is for ABI
conformance.

There are other ABIs on ARM that don't suffer from these problems.  We
should investigate using them.

: The point being that programmers *do* code with certain
: assumptions and as soon as those assumptions don't hold on
: a platform, you end up worse off. My thoughts for MIPS n32
: are to make it behave like any "normal" 32-bit strong-
: alignment platform to avoid 1) a large number of runtime
: alignment faults -- which are a bigger performance bottleneck
: than forcing 64-bit integrals to be accessed with 2 32-bit
: accesses

Run time faults aren't a bottleneck.  They are a core dump, which is
why I'm looking at this in the first place. :)

: and 2) avoid further abuse of __packed, which turns
: all accesses in a series of byte-wise accesses.

We haven't really abused __packed.  The items we have as '__packed'
really are packed data structures for the wire.  We have also added
some __aligned() flag as well which help to mitigate the performance
penalties because the compiler can unpack the items.

: At Juniper I changed the ARM compiler default by adding:
: 	-mstructure-size-boundary=8
: 
: That made life a *lot* simpler and performance hasn't been
: sacrificed.

Except you've invented a new ABI by doing that...  I think that the
project should look at transitioning to a different ABI that works
better.  ARM has several to choose from...

: Just an explanation of where I'm coming from...

Yes.  Unfortunately, those kinds of hacks take us further away form
the standard environment for these platforms.  There are many dark
corners of the current MIPS code that knows all the alignment and
layout issues and changing the compiler to force a different alignment
will break that code...

Anyway, since we are a little ways away from having to cope with all
the ABI issues for MIPS...

It also turns out that in this case, a simple (void *) is safe and
causes no issues because that time_t isn't accessed...  It does give
one time to pause and think about it.

Warner