Re: New syscons bugs: shutdown -r doesn't execute rc.d sequence and others

On Thu, 30 Mar 2017, Andrey Chernov wrote:

> On 30.03.2017 14:23, Andriy Gapon wrote:
>> On 30/03/2017 12:34, Andrey Chernov wrote:
>>> On 30.03.2017 12:23, Andrey Chernov wrote:
>>>> Yes, only for reboot/shutdown. The system does not do anythings wrong
>>>> even under high load. On reboot or hang those lines are never printed:
>>>>
>>>> kernel: Waiting (max 60 seconds) for system process `vnlru' to stop...done
>>>> kernel: Waiting (max 60 seconds) for system process `bufdaemon' to
>>>> stop...done
>>>> kernel: Waiting (max 60 seconds) for system process `syncer' to stop...
>>>> kernel: Syncing disks, vnodes remaining...5 3 0 1 0 0 done
>>>> kernel: All buffers synced.
>>>> (it is from 10-stable sample, old -current samples are lost)
>>>>
>>>> Moreover, GELI swap deactivation lines are never printed too (I already
>>>> mention that I change swap to normal, but nothing is changed).
>>>
>>> I start to have raw guess that _any_ kernel printf in shutdown mode
>>> cause not printf but premature reboot.
>>
>> This sounds somewhat familiar...
>> I vaguely recall an opposite issue that happened in the past.  After one of my
>> changes the reboot started hanging for one user.  Turned out that the actual bug
>> was always there, but previously the system rebooted because of a printf that
>> caused a LOR (between spinlocks, AFAIR), witness tried to report it... using
>> printf, and that recursed and there was a triple fault in the end.
>>
>> Let me try to dig some details, maybe the current issue is related in some ways.
>>
>> By chance, do you have WITNESS but not WITNESS_SKIPSPIN in your kernel config?
>
> No, I don't have WITNESS*
> I think removing all vt* lines from the kernel confing (and leaving sc)
> will be enough to reproduce it, but I am not sure.

INVARIANTS with WITNESS is not a bad way to debug problems :-).  I just
remembered to try it with recent changes.  It didn't find any problems
for rebooting.

The problems reported in Andriy's 2012 threads are almost exactly the
ones that I have mostly fixed in syscons -- LORs and deadlocks, and
endless recursion in WITNESS to report the problem.  Syscons now detects
and handles most LORs and deadlocks in itself, but I haven't committed
the fixes for upper layers yet, so syscons mostly doesn't get called.
cnputs() was "fixed" to silently drop the output.

There is still an annoying LOR for devfs vs ufs in reboot.  This is
reported with no problems since it is not related to consoles.

Bruce