Re: Stop scheduler on panic

On Mon, Nov 14, 2011 at 12:06:48PM +0200, Andriy Gapon wrote:
> on 13/11/2011 10:32 Kostik Belousov said the following:
> > I was tricked into finishing the work by Andrey Gapon, who developed the
> > patch to reliably stop other processors on panic.  The patch greatly
> > improves the chances of getting dump on panic on SMP host. Several people
> > already saw the patchset, and I remember that Andrey posted it to some
> > lists.
> > 
> > The change stops other (*) processors early upon the panic.  This way, no
> > parallel manipulation of the kernel memory is performed by CPUs. In
> > particular, the kernel memory map is static.  Patch prevents the panic
> > thread from blocking and switching out.
> > 
> > * - in the context of the description, other means not current.
> > 
> > Since other threads are not run anymore, lock owner cannot release a lock
> > which is required by panic thread.  Due to this, we need to fake a lock
> > acquisition after the panic, which adds minimal overhead to the locking
> > cost. The patch tries to not add any overhead on the fast path of the lock
> > acquire.  The check for the after-panic condition was reduced to single
> > memory access, done only when the quick cas lock attempt failed, and braced
> > with __unlikely compiler hint.
> > 
> > For now, the new mode of operation is disabled by default, since some 
> > further USB changes are needed to make USB keyboard usable in that 
> > environment.
> > 
> > With the patch, getting a dump from the machine without debugger compiled
> > in is much more realistic.  Please comment, I will commit the change in 2
> > weeks unless strong reasons not to are given.
> > 
> > http://people.freebsd.org/~kib/misc/stop_cpus_on_panic.1.patch
> > 
> 
> 
> On a more serious note:
> - some code in my latest version of the patch was contributed by or was based
> on the code or ideas contributed by jhb and mdf (so that attributions are not
> lost)
Please provide me with proper attribution for contributors and testers.

> - there was a concern about how sync-on-panic would work
> 
> About the latter, I have never really tested it. mdf has suggested to
> move the sync-on-panic code to a place after we ensure that there is
> only one CPU in panic(), but before we stop other CPUs.
sync_on_panic is incompatible with the patch. I argue that it provides
non-zero chance of damaging good filesystems even if panic was unrelated
to the fs/bio/device layer. As an example, consider the case when other
CPU was modifying in-memory representation of the metadata, and panic
happen on this CPU. If you write half-changed block back, you make more
damage to the filesystem then if you do not. The half-backed sync spoils
any journaling or SU consistency guarantees.

The issues of multithreading nature of our storage subsystem are secondary.

The user who sets either tunable shall know what he does.
>
> I think that I've already sent you a list of the early testers for
> various WIP versions of the patch.
I do not have the list.

BTW, if you want, feel free to handle the commit youself. You definitely
spent much more efforts on the stuff and deserve the credit.

I was promised in private that a review will be provided during this
weekend.