_at_r324591: panic: UNR inconsistency: items 0 found 7 (line 361)

From: David Wolfskill <david_at_catwhisker.org>
Date: Fri, 13 Oct 2017 04:36:34 -0700
This occurred after I had booted & smoke-tested my laptop, then
issued "sudo shutdown -r now"; it's *possible* that there was also
a (similar?) problem shutting down yesterday (_at_r324542), but the
screen had gone dark and declined to shed any light, so I'm not
sure on that point.

uname strings (yesterday & today):

FreeBSD g1-252.catwhisker.org 12.0-CURRENT FreeBSD 12.0-CURRENT #429  r324542M/324546:1200051: Thu Oct 12 05:09:28 PDT 2017     root_at_g1-252.catwhisker.org:/common/S4/obj/usr/src/sys/CANARY  amd64

FreeBSD g1-252.catwhisker.org 12.0-CURRENT FreeBSD 12.0-CURRENT #430  r324591M/324591:1200051: Fri Oct 13 03:58:11 PDT 2017     root_at_g1-252.catwhisker.org:/common/S4/obj/usr/src/sys/CANARY  amd64

Panic message, kernel buffer & stack trace from today:

panic: UNR inconsistency: items 0 found 7 (line 361)

Unread portion of the kernel message buffer:
<118>Writing entropy file:.
<118>Writing early boot entropy file:.
<118>.
<118>Terminated
<118>Oct 13 11:09:19 g1-252 syslogd: exiting on signal 15
<5>wlan0: link state changed to DOWN
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop... 
Syncing disks, vnodes remaining... 10 9 9 5 2 2 1 1 1 0 0 0 0 0 done
All buffers synced.
lock order reversal:
 1st 0xfffff8000efdd5f0 ufs (ufs) _at_ /usr/src/sys/kern/vfs_mount.c:1274
 2nd 0xfffff8000efdd068 syncer (syncer) _at_ /usr/src/sys/kern/vfs_subr.c:2768
stack backtrace:
#0 0xffffffff80a95693 at witness_debugger+0x73
#1 0xffffffff80a95512 at witness_checkorder+0xe02
#2 0xffffffff80a07e9e at lockmgr_lock_fast_path+0x1ae
#3 0xffffffff80fb2c20 at VOP_LOCK1_APV+0xe0
#4 0xffffffff80b0ed26 at _vn_lock+0x66
#5 0xffffffff80afdfd6 at vputx+0x156
#6 0xffffffff80af5a38 at dounmount+0x4d8
#7 0xffffffff80aff75b at vfs_unmountall+0x6b
#8 0xffffffff80adb6a3 at bufshutdown+0x393
#9 0xffffffff80a31b49 at kern_reboot+0x189
#10 0xffffffff80a31964 at sys_reboot+0x3c4
#11 0xffffffff80e4ab5b at amd64_syscall+0x7ab
#12 0xffffffff80e29b5b at Xfast_syscall+0xfb
lock order reversal:
 1st 0xfffff8000efde7c8 devfs (devfs) _at_ /usr/src/sys/kern/vfs_mount.c:1274
 2nd 0xfffff8000efde240 syncer (syncer) _at_ /usr/src/sys/kern/vfs_subr.c:2768
stack backtrace:
#0 0xffffffff80a95693 at witness_debugger+0x73
#1 0xffffffff80a95512 at witness_checkorder+0xe02
#2 0xffffffff80a07e9e at lockmgr_lock_fast_path+0x1ae
#3 0xffffffff80fb2c20 at VOP_LOCK1_APV+0xe0
#4 0xffffffff80b0ed26 at _vn_lock+0x66
#5 0xffffffff80afdfd6 at vputx+0x156
#6 0xffffffff80af5a38 at dounmount+0x4d8
#7 0xffffffff80aff75b at vfs_unmountall+0x6b
#8 0xffffffff80adb6a3 at bufshutdown+0x393
#9 0xffffffff80a31b49 at kern_reboot+0x189
#10 0xffffffff80a31964 at sys_reboot+0x3c4
#11 0xffffffff80e4ab5b at amd64_syscall+0x7ab
#12 0xffffffff80e29b5b at Xfast_syscall+0xfb
lock order reversal:
 1st 0xfffff8000ee3b7c8 ufs (ufs) _at_ /usr/src/sys/kern/vfs_mount.c:1274
 2nd 0xfffff8000ee3bd50 devfs (devfs) _at_ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1414
stack backtrace:
#0 0xffffffff80a95693 at witness_debugger+0x73
#1 0xffffffff80a95512 at witness_checkorder+0xe02
#2 0xffffffff80a07e9e at lockmgr_lock_fast_path+0x1ae
#3 0xffffffff80fb2c20 at VOP_LOCK1_APV+0xe0
#4 0xffffffff80b0ed26 at _vn_lock+0x66
#5 0xffffffff80d3c5f3 at ffs_flushfiles+0x93
#6 0xffffffff80d1fef2 at softdep_flushfiles+0x82
#7 0xffffffff80d3ebc7 at ffs_unmount+0x77
#8 0xffffffff80af5a78 at dounmount+0x518
#9 0xffffffff80aff75b at vfs_unmountall+0x6b
#10 0xffffffff80adb6a3 at bufshutdown+0x393
#11 0xffffffff80a31b49 at kern_reboot+0x189
#12 0xffffffff80a31964 at sys_reboot+0x3c4
#13 0xffffffff80e4ab5b at amd64_syscall+0x7ab
#14 0xffffffff80e29b5b at Xfast_syscall+0xfb
panic: UNR inconsistency: items 0 found 7 (line 361)

cpuid = 0
time = 1507892985
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff803a762b = db_trace_self_wrapper+0x2b/frame 0xfffffe0ba17c0620
vpanic() at 0xffffffff80a3234c = vpanic+0x19c/frame 0xfffffe0ba17c06a0
kassert_panic() at 0xffffffff80a321a6 = kassert_panic+0x126/frame 0xfffffe0ba17c0710
check_unrhdr() at 0xffffffff80a8ea10 = check_unrhdr+0x230/frame 0xfffffe0ba17c0760
delete_unrhdr() at 0xffffffff80a8eb13 = delete_unrhdr+0x13/frame 0xfffffe0ba17c0780
tmpfs_free_tmp() at 0xffffffff83214a4f = tmpfs_free_tmp+0x6f/frame 0xfffffe0ba17c07a0
tmpfs_unmount() at 0xffffffff832152b0 = tmpfs_unmount+0x1f0/frame 0xfffffe0ba17c07f0
dounmount() at 0xffffffff80af5a78 = dounmount+0x518/frame 0xfffffe0ba17c0860
vfs_unmountall() at 0xffffffff80aff75b = vfs_unmountall+0x6b/frame 0xfffffe0ba17c0890
bufshutdown() at 0xffffffff80adb6a3 = bufshutdown+0x393/frame 0xfffffe0ba17c08e0
kern_reboot() at 0xffffffff80a31b49 = kern_reboot+0x189/frame 0xfffffe0ba17c0920
sys_reboot() at 0xffffffff80a31964 = sys_reboot+0x3c4/frame 0xfffffe0ba17c0980
amd64_syscall() at 0xffffffff80e4ab5b = amd64_syscall+0x7ab/frame 0xfffffe0ba17c0ab0
Xfast_syscall() at 0xffffffff80e29b5b = Xfast_syscall+0xfb/frame 0xfffffe0ba17c0ab0
--- syscall (55, FreeBSD ELF64, sys_reboot), rip = 0x40ebfa, rsp = 0x7fffffffe708, rbp = 0x7fffffffe7f0 ---
KDB: enter: panic

__curthread () at ./machine/pcpu.h:232
232             __asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) #0  __curthread () at ./machine/pcpu.h:232
#1  doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:318
#2  0xffffffff803a498b in db_dump (dummy=<optimized out>, 
    dummy2=<unavailable>, dummy3=<unavailable>, dummy4=<unavailable>)
    at /usr/src/sys/ddb/db_command.c:572
#3  0xffffffff803a4749 in db_command (last_cmdp=<optimized out>, 
    cmd_table=<optimized out>, dopager=<optimized out>)
    at /usr/src/sys/ddb/db_command.c:479
#4  0xffffffff803a44e4 in db_command_loop ()
    at /usr/src/sys/ddb/db_command.c:532
#5  0xffffffff803a777f in db_trap (type=<optimized out>, code=<optimized out>)
    at /usr/src/sys/ddb/db_main.c:248
#6  0xffffffff80a75813 in kdb_trap (type=3, code=-61456, tf=<optimized out>)
    at /usr/src/sys/kern/subr_kdb.c:654
#7  0xffffffff80e495db in trap (frame=0xfffffe0ba17c0550)
    at /usr/src/sys/amd64/amd64/trap.c:536
#8  <signal handler called>
#9  kdb_enter (why=0xffffffff813999e1 "panic", msg=<optimized out>)
    at /usr/src/sys/kern/subr_kdb.c:444
#10 0xffffffff80a32369 in vpanic (fmt=<optimized out>, ap=0xfffffe0ba17c06e0)
    at /usr/src/sys/kern/kern_shutdown.c:772
#11 0xffffffff80a321a6 in kassert_panic (
    fmt=0xffffffff813a341e "UNR inconsistency: items %u found %u (line %d)\n")
    at /usr/src/sys/kern/kern_shutdown.c:669
#12 0xffffffff80a8ea10 in check_unrhdr (uh=<optimized out>, line=361)
    at /usr/src/sys/kern/subr_unit.c:246
#13 0xffffffff80a8eb13 in delete_unrhdr (uh=0xfffff8000ec78500)
    at /usr/src/sys/kern/subr_unit.c:361
#14 0xffffffff83214a4f in tmpfs_free_tmp (tmp=0xfffff8000eee0900)
    at /usr/src/sys/fs/tmpfs/tmpfs_vfsops.c:347
#15 0xffffffff832152b0 in tmpfs_unmount (mp=0xfffff8000ef6d000, 
    mntflags=<optimized out>) at /usr/src/sys/fs/tmpfs/tmpfs_vfsops.c:323
#16 0xffffffff80af5a78 in dounmount (mp=0xffffffff81b89ec0 <cnputs_mtx+24>, 
    flags=524288, td=0xfffff8000a8d7560) at /usr/src/sys/kern/vfs_mount.c:1369
#17 0xffffffff80aff75b in unmount_or_warn (mp=<optimized out>)
    at /usr/src/sys/kern/vfs_subr.c:4028
#18 vfs_unmountall () at /usr/src/sys/kern/vfs_subr.c:4062
#19 0xffffffff80adb6a3 in bufshutdown (show_busybufs=1)
    at /usr/src/sys/kern/vfs_bio.c:1305
#20 0xffffffff80a31b49 in kern_reboot (howto=0)
    at /usr/src/sys/kern/kern_shutdown.c:372
#21 0xffffffff80a31964 in sys_reboot (td=0xfffff8000a8d7560, 
    uap=0xfffff8000a8d7910) at /usr/src/sys/kern/kern_shutdown.c:248
#22 0xffffffff80e4ab5b in syscallenter (td=0xfffff8000a8d7560)
    at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:132
#23 amd64_syscall (td=0xfffff8000a8d7560, traced=0)
    at /usr/src/sys/amd64/amd64/trap.c:917
#24 <signal handler called>
#25 0x000000000040ebfa in ?? ()
Backtrace stopped: Cannot access memory at address 0x7fffffffe708
(kgdb) 

[Above cribbed from core.txt.2.]

I will copy the core and a few other possibly-interesting bits to
<http://www.catwhisker.org/~david/FreeBSD/head/r324591/>; other
bits that may be of interest (e.g., verbose dmesg.boot) may be found
from <http://www.catwhisker.org/~david/FreeBSD/history/>.  [This
is residential ADSL, so it's not high-bandwidth, especially for
data going from me to you.]

Here's what's in place:
albert(11.1-S)[12] dirs
~/public_html/FreeBSD/head/r324591 
albert(11.1-S)[13] ls -lTh
total 423949
-rw-r--r--  1 david  staff   155K Oct 13 04:14:46 2017 core.txt.2
-rw-r--r--  1 david  staff    36K Oct 13 04:14:46 2017 core.txt.2.gz
-rw-r--r--  1 david  staff   513B Oct 13 04:14:30 2017 info.2
-rw-r--r--  1 david  staff   360B Oct 13 04:14:30 2017 info.2.gz
-rw-r--r--  1 david  staff   860M Oct 13 04:14:34 2017 vmcore.2
-rw-r--r--  1 david  staff   157M Oct 13 04:14:34 2017 vmcore.2.gz
albert(11.1-S)[14] 

I note that my build machine (kept in lock-step for SVN revisions,
but running a GENERIC kernel) had no hint of an issue -- either
yesterday or today.

Peace,
david
-- 
David H. Wolfskill				david_at_catwhisker.org
Unsubstantiated claims of "Fake News" are evidence that the claimant lies again.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

Received on Fri Oct 13 2017 - 09:36:42 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:13 UTC