Re: svn commit: r313268 - head/sys/kern [through -r313271 for atomic_fcmpset use and later: fails on PowerMac G5 "Quad Core"; -r313266 works]

On 2017-Feb-24, at 4:23 PM, Mateusz Guzik <mjguzik at gmail.com> wrote:
> 
> On Tue, Feb 21, 2017 at 01:37:25AM -0800, Mark Millard wrote:
>> [Back to the powerpc64 context.]
>> 
>> On 2017-Feb-20, at 11:10 AM, Mateusz Guzik <mjguzik at gmail.com> wrote:
>> 
>>> On Sat, Feb 18, 2017 at 04:18:05AM -0800, Mark Millard wrote:
>>>> [Note: I experiment with clang based powerpc64 builds,
>>>> reporting problems that I find. Justin is familiar
>>>> with this, as is Nathan.]
>>>> 
>>>> I tried to update the PowerMac G5 (a so-called "Quad Core")
>>>> that I have access to from head -r312761 to -r313864 and
>>>> ended up with random panics and hang ups in fairly short
>>>> order after booting.
>>>> 
>>>> Some approximate bisecting for the kernel lead to:
>>>> (sometimes getting part way into a buildkernel attempt
>>>> for a different version before a failure happens)
>>>> 
>>>> -r313266: works (just before use of atomic_fcmpset)
>>>> vs.
>>>> -r313271: fails (last of the "use atomic_fcmpset" check-ins)
>>>> 
>>>> (I did not try -r313268 through -r313270 as the use was
>>>> gradually added.)
>>>> 
>>>> So I'm currently running a -r313864 world with a -r313266
>>>> kernel.
>>>> 
>>>> No kernel that I tried that was from before -r313266 had the
>>>> problems.
>>>> 
>>>> Any kernel that I tried that was from after -r313271 had the
>>>> problems.
>>>> 
>>>> Of course I did not try them all in other direction. :)
>>>> 
>>> 
>>> I found that spin mutexes were not properly handling this, fixed in
>>> r313996.
>>> 
>>> Locally I added a if (cpu_tick() % 2) return (0); snipped to amd64
>>> fcmpset to simulate failures. Everything works, while it would easily
>>> fail without the patch.
>>> 
>>> That said, I hope this concludes the 'missing check for not-reread value
>>> of failed fcmpset' saga.
>>> 
>>> -- 
>>> Mateusz Guzik <mjguzik gmail.com>
>> 
>> -r313999 is an improvement for powerpc64: it boots and I can
>> log in on the old PowerMac G5 so-called "Quad Core".
>> 
>> But, e.g., buildworld buildkernel eventually hangs and later
>> the powerpc64 panics for "spin lock held too long".
>> 
> 
> Allright, play time is over.
> 
> Can you please:
> 1. verify r313254 is stable for you
> 2. apply https://people.freebsd.org/~mjg/patches/complete-locks.diff and
> https://people.freebsd.org/~mjg/.junk/ppc.diff on top of it and retry
> the test?
> 
> This is a workaround which effectively disables the powerpc-specific
> primitive and makes it use a cmpset wrapper instead. I don't have the
> hardware to test right now and my attempts to boot in qemu also failed.
> 
> That said, does not look like there are general fcmpset bugs left and
> the remaining issue seems powerpc-specific.
> 
> If this works, I'll commit the workaround for the time being as in few
> weeks I'd like to start merging the work back to stable/11.
> 
> -- 
> Mateusz Guzik <mjguzik gmail.com>

I've started a self-hosted powerpc64 -r313254 build
based on running the -r313266 kernel. (The context 
sometimes do cross builds in is tied up with other
things. -r313266 is what my prior bisection came up
with as the last appearently-working kernel at the
time.)

So it will be a while before I have a -r313254 in
place to try: the self-hosted build takes longer
and so will not be installed for a while.

To judge stability I'll probably have -e313254 build
the patched update that you want me to test, initially
doing a cleanworld. So that too will take a while.

(The above wording presumes all goes well.)

I'll let you know as I go along if I run into anything
interesting.


My builds are rebuilding both world and kernel since
what turns into /usr/include/sys/* has changes in your
patch.

The builds are without MALLOC_PRODUCTION but are
otherwise not debug builds.


I've not seen anything indicating that anyone has
been trying TARGET_ARCH=powerpc. I've been trying
TARGET_ARCH=powerpc64 .

While I do not have access to a true
TARGET_ARCH=powerpc machine currently, such a build
can be used on a PowerMac G5 so-called "Quad Core".
So I could eventually build and try such on the one
powerpc family machine that I currently have access
to.

clang 3.9.1 has a significant code generation problem
for TARGET_ARCH=powerpc and so I'd have to use
a gcc 4.2.1 based build for that sort of experiment.
(There is no xtoolchain for 32-bit powerpc.)

I use clang 3.9.1 or xtoolchain for
TARGET_ARCH=powerpc64 and have been using clang 3.9.1
in recent times. My primary powerpc family use has
been to experiment with building based on the
modern libc++ and reporting issues discovered in the
attempts. This explains the clang/xtoolchain context.

clang 3.9.1 has major problems for C++ exception
handling for both powerpc64 and powerpc but a
lot of FreeBSD is independent of throwing C++
exceptions. By contrast xtoolchain-based works
for C++ exception handling but lib32 fails
to operate when built by a xtoolchain build.

===
Mark Millard
markmi at dsl-only.net