Re: Page fault in udp_usrreq.c:823

From: Cy Schubert <Cy.Schubert_at_cschubert.com>
Date: Thu, 21 Jun 2018 18:14:52 -0700
In message <20180620090957.GA130_at_x2.osted.lan>, Peter Holm writes:
> 20180620 10:32:47 all (1/1): udp.sh
> Kernel page fault with the following non-sleepable locks held:
> shared rw udpinp (udpinp) r = 0 (0xfffff80bbc808d78) locked _at_ netinet/in_pcb.
> c:2398
> stack backtrace:
> #0 0xffffffff80c00733 at witness_debugger+0x73
> #1 0xffffffff80c01b11 at witness_warn+0x461
> #2 0xffffffff81075763 at trap_pfault+0x53
> #3 0xffffffff81074d7a at trap+0x2ba
> #4 0xffffffff8105076c at calltrap+0x8
> #5 0xffffffff80dd21b0 at udp_ctlinput+0x50
> #6 0xffffffff80d3081d at icmp_input+0x96d
> #7 0xffffffff80d316d7 at ip_input+0x3f7
> #8 0xffffffff80cc0a92 at netisr_dispatch_src+0xa2
> #9 0xffffffff80ca3ebe at ether_demux+0x16e
> #10 0xffffffff80ca5377 at ether_nh_input+0x427
> #11 0xffffffff80cc0a92 at netisr_dispatch_src+0xa2
> #12 0xffffffff80ca437f at ether_input+0x8f
> #13 0xffffffff80cbc500 at iflib_rxeof+0xc90
> #14 0xffffffff80cb6b6f at _task_fn_rx+0x7f
> #15 0xffffffff80bdd209 at gtaskqueue_run_locked+0x139
> #16 0xffffffff80bdcf88 at gtaskqueue_thread_loop+0x88
> #17 0xffffffff80b54514 at fork_exit+0x84
>
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 10; apic id = 0a
> fault virtual address = 0x8
> fault code  = supervisor read data, page not present
> instruction pointer = 0x20:0xffffffff80dd2423
> stack pointer         = 0x0:0xfffffe00004a5500
> frame pointer         = 0x0:0xfffffe00004a55a0
> code segment  = base 0x0, limit 0xfffff, type 0x1b
>    = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags = interrupt enabled, resume, IOPL = 0
> current process  = 0 (if_io_tqg_10)
> [ thread pid 0 tid 100069 ]
> Stopped at      udp_common_ctlinput+0x263:      cmpq    $0,0x8(%rax)
> db>
>
> Details _at_ https://people.freebsd.org/~pho/stress/log/udp_usrreq.txt
>
> -- 
> Peter
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>

This is surprisingly similar to my panic. Twice since June 19.

slippy# kgdb /boot/kernel/kernel vmcore.3 
GNU gdb (GDB) 8.1 [GDB v8.1 for FreeBSD]
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.
html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show 
copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd12.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /boot/kernel/kernel...Reading symbols from 
/usr/lib/debug//boot/kernel/kernel.debug...done.
done.

Unread portion of the kernel message buffer:
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-CURRENT #355 r335477M: Thu Jun 21 05:26:35 PDT 2018
    root_at_slippy:/export/obj/opt/src/svn-current/amd64.amd64/sys/BREAK 
amd64
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on 
LLVM 6.0.0)
VT(vga): text 80x25
module_register: cannot register mmc/mmcsd from kernel; already loaded 
from mmcsd.ko
Module mmc/mmcsd failed to register: 17
CPU: Intel(R) Core(TM) i3-2350M CPU _at_ 2.30GHz (2294.83-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206a7  Family=0x6  Model=0x2a  Stepping=7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,P
GE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x1dbae3bf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE
3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,XSAVE,OSXSAVE,A
VX>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 8080965632 (7706 MB)
ACPI APIC Table: <ACRSYS ACRPRDCT>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
random: entropy device external interface
module_register_init: MOD_LOAD (vesa, 0xffffffff8097b700, 0) error 19
kbd1 at kbdmux0
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <ACRSYS ACRPRDCT> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
acpi_ec0: <Embedded Controller: GPE 0x17> port 0x62,0x66 on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0x2000-0x203f mem 
0xf0000000-0xf03fffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
vgapci0: Boot video device
pci0: <simple comms> at device 22.0 (no driver attached)
ehci0: <Intel Cougar Point USB 2.0 controller> mem 
0xf060a000-0xf060a3ff irq 16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
hdac0: <Intel Cougar Point HDA Controller> mem 0xf0600000-0xf0603fff 
irq 22 at device 27.0 on pci0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
pci2: <ACPI PCI bus> on pcib2
iwn0: <Intel Centrino Advanced-N 6205> mem 0xf0500000-0xf0501fff irq 17 
at device 0.0 on pci2
pcib3: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci3: <ACPI PCI bus> on pcib3
bge0: <Broadcom BCM57765 B0, ASIC rev. 0x57785100> mem 
0xf0400000-0xf040ffff,0xf0410000-0xf041ffff irq 19 at device 0.0 on pci3
bge0: CHIP ID 0x57785100; ASIC REV 0x57785; CHIP REV 0x577851; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM57765 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 
1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
<6>bge0: Using defaults for TSO: 65518/35/2048
<6>bge0: Ethernet address: 20:6a:8a:72:03:17
pci3: <base peripheral, SD host controller> at device 0.1 (no driver 
attached)
ehci1: <Intel Cougar Point USB 2.0 controller> mem 
0xf0609000-0xf06093ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Cougar Point AHCI SATA controller> port 
0x2088-0x208f,0x2094-0x2097,0x2080-0x2087,0x2090-0x2093,0x2060-0x207f 
mem 0xf0608000-0xf06087ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
ichsmb0: <Intel Cougar Point SMBus controller> port 0xefa0-0xefbf mem 
0xf0604000-0xf06040ff irq 18 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
smb0: <SMBus generic I/O> on smbus0
acpi_lid0: <Control Method Lid Switch> on acpi0
acpi_button0: <Sleep Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model Synaptics Touchpad, device ID 0
acpi_acad0: <AC Adapter> on acpi0
battery0: <ACPI Control Method Battery> on acpi0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid 
PNP0900 on isa0
ata0: <ATA channel> at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
ata1: <ATA channel> at port 0x170-0x177,0x376 irq 15 on isa0
acpi_perf0: <ACPI CPU Frequency Control> on cpu0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 10.000 msec
IP Filter: v5.1.2 initialized.  Default = pass all, Logging = enabled
hdacc0: <Realtek ALC269 HDA CODEC> at cad 0 on hdac0
hdaa0: <Realtek ALC269 Audio Function Group> at nid 1 on hdacc0
pcm0: <Realtek ALC269 (Analog 2.0+HP/2.0)> at nid 20,33 and 27 on hdaa0
pcm1: <Realtek ALC269 (Left Analog Mic)> at nid 24 on hdaa0
hdacc1: <Intel Cougar Point HDA CODEC> at cad 3 on hdac0
hdaa1: <Intel Cougar Point Audio Function Group> at nid 1 on hdacc1
pcm2: <Intel Cougar Point (HDMI/DP 8ch)> at nid 5 on hdaa1
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen1.1: <Intel EHCI root HUB> at usbus1
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen0.1: <Intel EHCI root HUB> at usbus0
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ses0 at ahciem0 bus 0 scbus3 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <HGST HTS721010A9E630 JB0OA3U0> ATA8-ACS SATA 3.x device
ada0: Serial Number JR1000D33969RE
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 953869MB (1953525168 512 byte sectors)
cd0 at ahcich1 bus 0 scbus1 target 0 lun 0
cd0: <PIONEER DVD-RW DVRTD11RS 1.01> Removable CD-ROM SCSI device
cd0: Serial Number SBB5103801
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 
8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present 
- tray closed
Launching APs: 1 3 2
Trying to mount root from ufs:/dev/ufs/Sroot [rw]...
Timecounter "TSC-low" frequency 1147416846 Hz quality 1000
<118>Setting hostuuid: 34f5ed40-8938-11da-b265-efe316da850d.
<118>Setting hostid: 0x7f5a03b9.
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
ugen0.2: <vendor 0x8087 product 0x0024> at usbus0
uhub2 on uhub1
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> 
on usbus0
ugen1.2: <vendor 0x8087 product 0x0024> at usbus1
uhub3 on uhub0
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> 
on usbus1
uhub2: 6 ports with 6 removable, self powered
uhub3: 6 ports with 6 removable, self powered
ugen0.3: <Logitech USB Receiver> at usbus0
ukbd0 on uhub2
ukbd0: <Logitech USB Receiver, class 0/0, rev 2.00/12.07, addr 3> on 
usbus0
kbd2 at ukbd0
ums0 on uhub2
ums0: <Logitech USB Receiver, class 0/0, rev 2.00/12.07, addr 3> on 
usbus0
ums0: 16 buttons and [XYZT] coordinates ID=2
ugen1.3: <vendor 0x05e3 USB2.0 Hub> at usbus1
uhub4 on uhub3
uhub4: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/32.98, addr 3> on 
usbus1
uhub4: 4 ports with 4 removable, self powered
ugen0.4: <AABBX0E6S 1.3M HD WebCam> at usbus0
ugen1.4: <vendor 0x05e3 USB2.0 Hub> at usbus1
uhub5 on uhub4
uhub5: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/32.98, addr 4> on 
usbus1
uhub5: 4 ports with 4 removable, self powered
ugen1.5: <ALCOR Generic USB Hub> at usbus1
uhub6 on uhub5
uhub6: <ALCOR Generic USB Hub, class 9/0, rev 1.10/3.12, addr 5> on 
usbus1
uhub6: 4 ports with 4 removable, self powered
ugen1.6: <Logitech USB Receiver> at usbus1
ukbd1 on uhub6
ukbd1: <Logitech USB Receiver, class 0/0, rev 2.00/12.07, addr 6> on 
usbus1
kbd3 at ukbd1
ums1 on uhub6
ums1: <Logitech USB Receiver, class 0/0, rev 2.00/12.07, addr 6> on 
usbus1
ums1: 16 buttons and [XYZT] coordinates ID=2
ugen1.7: <Device 2Port KVMSwitcher> at usbus1
ukbd2 on uhub6
ukbd2: <Device 2Port KVMSwitcher, class 0/0, rev 1.10/0.01, addr 7> on 
usbus1
kbd4 at ukbd2
ugen1.8: <vendor 0x05e3 USB TO IDE> at usbus1
umass0 on uhub5
umass0: <vendor 0x05e3 USB TO IDE, class 0/0, rev 2.00/0.33, addr 8> on 
usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x408c
umass0:6:0: Attached to scbus6
da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
da0: <TOSHIBA MK8026GAX 0811> Fixed Direct Access SCSI device
da0: 40.000MB/s transfers
da0: 76319MB (156301488 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
<118>Starting file system checks:
<118>/dev/ufs/Sroot: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/Sroot: clean, 413829 free (909 frags, 51615 blocks, 0.2% 
fragmentation)
<118>/dev/ufs/Susr: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/Susr: clean, 459287 free (4847 frags, 56805 blocks, 0.6% 
fragmentation)
<118>/dev/ufs/Svtmp: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/Svtmp: clean, 503642 free (50 frags, 62949 blocks, 0.0% 
fragmentation)
<118>/dev/ufs/SCusr: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/Svar: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/SSusr: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/SCusr: clean, 469826 free (4578 frags, 58156 blocks, 0.6% 
fragmentation)
<118>/dev/ufs/Svar: clean, 453726 free (574 frags, 56644 blocks, 0.1% 
fragmentation)
<118>/dev/ufs/SCroot: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/SSusr: clean, 569245 free (3653 frags, 70699 blocks, 0.5% 
fragmentation)
<118>/dev/ufs/SCroot: clean, 422625 free (617 frags, 52751 blocks, 0.1% 
fragmentation)
<118>/dev/ufs/SCvar: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/SSvar: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/SCvar: clean, 400011 free (99 frags, 49989 blocks, 0.0% 
fragmentation)
<118>/dev/ufs/SSvar: clean, 457058 free (138 frags, 57115 blocks, 0.0% 
fragmentation)
<118>/dev/ufs/SSroot: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/ufs/SSroot: clean, 412995 free (219 frags, 51597 blocks, 0.0% 
fragmentation)
<118>/dev/msdosfs/SHARED: FILESYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/i386/disk0s1a: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/i386/disk0s1a: clean, 254465 free (337 
frags, 31766 blocks, 0.1% fragmentation)
<118>/dev/zvol/tank/VMs/stable10/amd64/disk0s1a: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable10/i386/disk0s1a: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable10/amd64/disk0s1a: clean, 335508 free 
(1348 frags, 41770 blocks, 0.1% fragmentation)
<118>/dev/zvol/tank/VMs/stable10/i386/disk0s1a: clean, 566833 free 
(2297 frags, 70567 blocks, 0.2% fragmentation)
<118>/dev/zvol/tank/VMs/stable11/amd64/disk0s1a: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/amd64/disk0s1a: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable11/amd64/disk0s1a: clean, 510529 free 
(1185 frags, 63668 blocks, 0.1% fragmentation)
<118>/dev/zvol/tank/VMs/current/amd64/disk0s1a: clean, 354656 free (336 
frags, 44290 blocks, 0.1% fragmentation)
<118>/dev/zvol/tank/VMs/stable11/i386/disk0s1a: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable11/i386/disk0s1a: clean, 633994 free 
(2106 frags, 78986 blocks, 0.2% fragmentation)
<118>/dev/zvol/tank/VMs/current/amd64/disk3p1: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/amd64/disk3p1: clean, 2535564 free (564 
frags, 316875 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/stable11/amd64/disk0s1d: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable11/amd64/disk0s1d: clean, 897924 free 
(188 frags, 112217 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/stable10/amd64/disk0s1d: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable10/amd64/disk0s1d: clean, 897896 free 
(152 frags, 112218 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/stable10/i386/disk0s1d: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable11/i386/disk0s1d: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable10/i386/disk0s1d: clean, 889422 free (214 
frags, 111151 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/stable11/i386/disk0s1d: clean, 889419 free (219 
frags, 111150 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/current/amd64/disk0s1d: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/amd64/disk0s1d: clean, 408410 free (90 
frags, 51040 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/stable10/amd64/disk0s1e: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable11/amd64/disk0s1e: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable10/amd64/disk0s1e: clean, 1230068 free 
(7300 frags, 152846 blocks, 0.5% fragmentation)
<118>/dev/zvol/tank/VMs/stable11/amd64/disk0s1e: clean, 726361 free 
(6297 frags, 90008 blocks, 0.4% fragmentation)
<118>/dev/zvol/tank/VMs/stable10/i386/disk0s1e: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable10/i386/disk0s1e: clean, 1299097 free 
(6385 frags, 161589 blocks, 0.4% fragmentation)
<118>/dev/zvol/tank/VMs/current/amd64/disk0s1e: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/amd64/disk0s1e: clean, 822303 free 
(1047 frags, 102657 blocks, 0.1% fragmentation)
<118>/dev/zvol/tank/VMs/current/i386/disk0s1d: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/i386/disk0s1d: clean, 399855 free (15 
frags, 49980 blocks, 0.0% fragmentation)
<118>/dev/zvol/tank/VMs/stable11/i386/disk0s1e: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/stable11/i386/disk0s1e: clean, 937324 free 
(7524 frags, 116225 blocks, 0.5% fragmentation)
<118>/dev/zvol/tank/VMs/current/i386/disk0s1e: FILE SYSTEM CLEAN; 
SKIPPING CHECKS
<118>/dev/zvol/tank/VMs/current/i386/disk0s1e: clean, 1077125 free (845 
frags, 134535 blocks, 0.1% fragmentation)
<118>Mounting local filesystems:.
<118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib 
/usr/local/lib/compat/pkg /usr/local/krb5/lib /usr/local/kde4/lib 
/usr/local/lib/compat /usr/local/lib/compat/pkg /usr/local/lib/dovecot 
/usr/local/lib/e2fsprogs /usr/local/lib/ffmpeg0 /usr/local/lib/freeradiu
s-3.0.15 /usr/local/lib/gcc48 /usr/local/lib/gcc5 /usr/local/lib/gcc6 
/usr/local/lib/gcc7 /usr/local/lib/gcc8 /usr/local/lib/gcc9 
/usr/local/lib/gegl-0.2 /usr/local/lib/graphviz /usr/local/lib/httrack 
/usr/local/lib/itcl3.4 /usr/local/lib/jitsi/lib/native 
/usr/local/lib/mysql /usr/local/lib/mysql/plugin /usr/local/lib/nss 
/usr/local/lib/opencollada /usr/local/lib/perl5/5.24/mach/CORE 
/usr/local/lib/perl5/5.26/mach/CORE /usr/local/lib/perl5/5.28/mach/CORE 
/usr/local/lib/pgtcl /usr/local/lib/pidgin /usr/local/lib/pth 
/usr/local/lib/qt4 /usr/local/lib/qt5 /usr/local/lib/samba4 
/usr/local/lib/xrdp /usr/local/libexec/openldap /usr/local/llvm-devel/li
b /usr/local/llvm40/lib /usr/local/llvm50/lib /usr/local/llvm60/lib 
/usr/local/share/chromium
<118>32-bit compatibility ldconfig path: /usr/lib32 
/alt/i386/root/usr/local/lib /usr/local/lib32/compat
<118>Setting hostname: slippy.
<118>Additional TCP/IP options: rfc1323 extensions=NO.
<118>Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ET
HER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
<118>Feeding entropy: .
<118>Loading IP Pools.
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>70018:add pool node(252.0.0.1/255.255.255.255: node entry already 
present in pool
<118>Enabling ipfilter.
<118>32:1:ioctl(add/insert rule): rule already exists
<118>Installing NAT rules.
<118>0 entries flushed from NAT table
<118>0 entries flushed from NAT list
<6>wlan0: Ethernet address: 20:6a:8a:72:03:17
<118>Created wlan(4) interfaces: wlan0.
<118>Created clone interfaces: lagg0.
<6>lo0: link state changed to UP
<6>bge0: link state changed to DOWN
iwn0: iwn_read_firmware: ucode rev=0x12a80601
<118>Starting wpa_supplicant.
<6>lagg0: link state changed to DOWN
iwn0: iwn_read_firmware: ucode rev=0x12a80601
<118>Starting dhclient.
<118>lagg0: no link ...
<6>wlan0: link state changed to UP
<6>lagg0: link state changed to UP
<118> got link
<6>bge0: link state changed to UP
<118>Starting Network: lo0 bge0 wlan0 lagg0.
<118>lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
<118>	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
<118>	inet 127.0.0.1 netmask 0xff000000 
<118>	inet6 ::1 prefixlen 128 
<118>	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
<118>	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
<118>	groups: lo 
<118>bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 
mtu 1500
<118>	options=80080<VLAN_HWCSUM,LINKSTATE>
<118>	ether 20:6a:8a:72:03:17
<118>	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
<118>	media: Ethernet autoselect (1000baseT <full-duplex>)
<118>	status: active
<118>wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 
mtu 1500
<118>	ether 20:6a:8a:72:03:17
<118>	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
<118>	media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
<118>	status: associated
<118>	ssid Walden_Pond channel 1 (2412 MHz 11g ht/20) bssid 
4c:60:de:31:3c:17
<118>	regdomain FCC country US authmode WPA2/802.11i privacy ON
<118>	deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 120 scanvalid 
16959
<118>	protmode CTS ampdulimit 64k ampdudensity 8 -amsdutx amsdurx 
shortgi
<118>	-stbc -ldpc wme roaming MANUAL
<118>	groups: wlan 
<118>lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 
mtu 1500
<118>	ether 20:6a:8a:72:03:17
<118>	inet6 fe80::226a:8aff:fe72:317%lagg0 prefixlen 64 scopeid 0x4 
<118>	inet 0.0.0.0 netmask 0x0 broadcast 255.255.255.255 
<118>	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
<118>	media: Ethernet autoselect
<118>	status: active
<118>	groups: lagg 
<118>	laggproto failover lagghash l2,l3,l4
<118>	laggport: bge0 flags=5<MASTER,ACTIVE>
<118>	laggport: wlan0 flags=0<>
<118>filter sync'd
<118>Starting devd.
<118>Starting ums0 moused.
<118>Autoloading module: uhid.ko
uhid0 on uhub2
uhid0: <Logitech USB Receiver, class 0/0, rev 2.00/12.07, addr 3> on 
usbus0
uhid1 on uhub6
uhid1: <Logitech USB Receiver, class 0/0, rev 2.00/12.07, addr 6> on 
usbus1
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>Cannot 'start' webcamd. Set webcamd_enable to YES in /etc/rc.conf 
or use 'onestart' instead of 'start'.
uhid2 on uhub6
uhid2: <Device 2Port KVMSwitcher, class 0/0, rev 1.10/0.01, addr 7> on 
usbus1
device_attach: uhid2 attach returned 12
<118>Cannot 'start' webcamd. Set webcamd_enable to YES in /etc/rc.conf 
or use 'onestart' instead of 'start'.
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>Starting ums1 moused.
<118>Autoloading module: uhid.ko
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>Autoloading module: uhid.ko
<118>Cannot 'start' uhidd. Set uhidd_enable to YES in /etc/rc.conf or 
use 'onestart' instead of 'start'.
<118>local: Not in a function
<118>local: Not in a function
<118>dhclient not running? (check /var/run/dhclient.bge0.pid).
<118>Stopping dhclient.
<118>Waiting for PIDS: 471.
<118>Starting dhclient.
<118>add host 127.0.0.1: gateway lo0 fib 0: route already in table
<118>Additional inet routing options: ignore ICMP redirect=YES log ICMP 
redirect=YES.
<118>add host ::1: gateway lo0 fib 0: route already in table
<118>add net fe80::: gateway ::1
<118>add net ff02::: gateway ::1
<118>add net ::ffff:0.0.0.0: gateway ::1
<118>add net ::0.0.0.0: gateway ::1
<118>Starting rtsold.


Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address	= 0x0
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80806966
stack pointer	        = 0x28:0xfffffe00004667d0
frame pointer	        = 0x28:0xfffffe0000466840
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi1: netisr 0)
trap number		= 12
panic: page fault
cpuid = 3
time = 1529587144
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
0xfffffe0000466480
vpanic() at vpanic+0x1a3/frame 0xfffffe00004664e0
panic() at panic+0x43/frame 0xfffffe0000466540
trap_fatal() at trap_fatal+0x35f/frame 0xfffffe0000466590
trap_pfault() at trap_pfault+0x49/frame 0xfffffe00004665f0
trap() at trap+0x29e/frame 0xfffffe0000466700
calltrap() at calltrap+0x8/frame 0xfffffe0000466700
--- trap 0xc, rip = 0xffffffff80806966, rsp = 0xfffffe00004667d0, rbp = 
0xfffffe0000466840 ---
udp_append() at udp_append+0x26/frame 0xfffffe0000466840
udp_input() at udp_input+0x574/frame 0xfffffe0000466910
ip_input() at ip_input+0x145/frame 0xfffffe0000466970
swi_net() at swi_net+0x17b/frame 0xfffffe00004669e0
intr_event_execute_handlers() at intr_event_execute_handlers+0xe9/frame 
0xfffffe0000466a20
ithread_loop() at ithread_loop+0xe7/frame 0xfffffe0000466a70
fork_exit() at fork_exit+0x83/frame 0xfffffe0000466ab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000466ab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
Uptime: 51s
Dumping 697 out of 7975 MB:..3%..12%..21%..33%..42%..51%..62%..72%..81%.
.92%

__curthread () at ./machine/pcpu.h:231
231		__asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) bt
#0  __curthread () at ./machine/pcpu.h:231
#1  doadump (textdump=1) at /opt/src/svn-current/sys/kern/kern_shutdown.
c:366
#2  0xffffffff8061b330 in kern_reboot (howto=260)
    at /opt/src/svn-current/sys/kern/kern_shutdown.c:446
#3  0xffffffff8061b7c3 in vpanic (fmt=<optimized out>, 
ap=0xfffffe0000466520)
    at /opt/src/svn-current/sys/kern/kern_shutdown.c:863
#4  0xffffffff8061b5b3 in panic (fmt=<unavailable>)
    at /opt/src/svn-current/sys/kern/kern_shutdown.c:790
#5  0xffffffff8095e76f in trap_fatal (frame=0xfffffe0000466710, eva=0)
    at /opt/src/svn-current/sys/amd64/amd64/trap.c:892
#6  0xffffffff8095e7c9 in trap_pfault (frame=0xfffffe0000466710, 
usermode=0)
    at /opt/src/svn-current/sys/amd64/amd64/trap.c:728
#7  0xffffffff8095ddee in trap (frame=0xfffffe0000466710)
    at /opt/src/svn-current/sys/amd64/amd64/trap.c:427
#8  <signal handler called>
#9  udp_append (inp=0xfffff80031d2ed58, ip=0xfffff800039333a0, 
    n=0xfffff80003933300, off=20, udp_in=0xfffffe0000466850)
    at /opt/src/svn-current/sys/netinet/udp_usrreq.c:314
#10 0xffffffff80806464 in udp_input (mp=<optimized out>, 
offp=<optimized out>,
    proto=17) at /opt/src/svn-current/sys/netinet/udp_usrreq.c:722
#11 0xffffffff8075f645 in ip_input (m=0x0)
    at /opt/src/svn-current/sys/netinet/ip_input.c:825
#12 0xffffffff8073959b in netisr_process_workstream_proto (
---Type <return> to continue, or q <return> to quit---q
nwsp=<optimized out>,Quit
(kgdb) frame 9
#9  udp_append (inp=0xfffff80031d2ed58, ip=0xfffff800039333a0, 
    n=0xfffff80003933300, off=20, udp_in=0xfffffe0000466850)
    at /opt/src/svn-current/sys/netinet/udp_usrreq.c:314
314		if (up->u_tun_func != NULL) {
(kgdb) l
309	
310		/*
311		 * Engage the tunneling protocol.
312		 */
313		up = intoudpcb(inp);
314		if (up->u_tun_func != NULL) {
315			in_pcbref(inp);
316			INP_RUNLOCK(inp);
317			(*up->u_tun_func)(n, off, inp, (struct sockaddr *)&udp_in[0],
318			    up->u_tun_ctx);
(kgdb) p up
$1 = (struct udpcb *) 0x0
(kgdb) p *inp
$2 = {inp_hash = {cle_next = 0x0, cle_prev = 0xfffff800023abf70}, 
  inp_pcbgrouphash = {cle_next = 0x0, cle_prev = 0x0}, inp_lock = {
    lock_object = {lo_name = 0xffffffff80a02fc7 "udpinp", lo_flags = 
90898432,
      lo_data = 0, lo_witness = 0x0}, rw_lock = 33}, inp_hpts = {
    tqe_next = 0x0, tqe_prev = 0x0}, inp_hpts_request = 0, 
  inp_in_hpts = 0 '\000', inp_in_input = 0 '\000', inp_hpts_cpu = 0, 
  inp_refcount = 1, inp_flags = 8405056, inp_flags2 = 16, inp_input_cpu 
= 0,
  inp_hpts_cpu_set = 0 '\000', inp_input_cpu_set = 0 '\000', 
  inp_hpts_calls = 0 '\000', inp_input_calls = 0 '\000', 
  inp_spare_bits2 = 0 '\000', inp_spare_byte = 0 '\000', inp_ppcb = 
0x0,
  inp_socket = 0x0, inp_hptsslot = 0, inp_hpts_drop_reas = 0, inp_input 
= {
    tqe_next = 0x0, tqe_prev = 0x0}, inp_pcbinfo = 0xfffffe0048566738, 
  inp_pcbgroup = 0x0, inp_pcbgroup_wild = {cle_next = 0x0, cle_prev = 
0x0},
  inp_cred = 0xfffff80031366600, inp_flow = 0, inp_vflag = 1 '\001', 
  inp_ip_ttl = 64 '_at_', inp_ip_p = 0 '\000', inp_ip_minttl = 0 '\000', 
  inp_flowid = 60, inp_snd_tag = 0x0, inp_flowtype = 63, 
  inp_rss_listen_bucket = 0, inp_inc = {inc_flags = 0 '\000', 
    inc_len = 0 '\000', inc_fibnum = 0, inc_ie = {ie_fport = 0, 
      ie_lport = 28353, ie_dependfaddr = {id46_addr = {ia46_pad32 = {0, 
0, 0},
          ia46_addr4 = {s_addr = 0}}, id6_addr = {__u6_addr = {
            __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 
0, 0, 0,
              0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}}, 
ie_dependladdr = {
        id46_addr = {ia46_pad32 = {0, 0, 0}, ia46_addr4 = {
---Type <return> to continue, or q <return> to quit---
            s_addr = 16777343}}, id6_addr = {__u6_addr = {
            __u6_addr8 = '\000' <repeats 12 times>, "\177\000\000\001", 
            __u6_addr16 = {0, 0, 0, 0, 0, 0, 127, 256}, __u6_addr32 = 
{0, 0,
              0, 16777343}}}}, ie6_zoneid = 0}}, inp_label = 0x0, 
  inp_sp = 0xfffff8000725dee0, {inp_ip_tos = 0 '\000', inp_options = 
0x0,
    inp_moptions = 0x0}, {in6p_options = 0x0, in6p_outputopts = 0x0, 
    in6p_moptions = 0x0, in6p_icmp6filt = 0x0, in6p_cksum = 0, 
in6p_hops = 0},
  inp_portlist = {cle_next = 0x0, cle_prev = 0xfffff80002334d20}, 
  inp_phd = 0xfffff80002334d00, inp_gencnt = 169, inp_lle = 0x0, 
  inp_rt_cookie = 7, {inp_route = {ro_rt = 0x0, ro_lle = 0x0, 
      ro_prepend = 0x0, ro_plen = 0, ro_flags = 256, ro_mtu = 0, spare 
= 0,
      ro_dst = {sa_len = 0 '\000', sa_family = 0 '\000', 
        sa_data = '\000' <repeats 13 times>}}, inp_route6 = {ro_rt = 
0x0,
      ro_lle = 0x0, ro_prepend = 0x0, ro_plen = 0, ro_flags = 256, 
ro_mtu = 0,
      spare = 0, ro_dst = {sin6_len = 0 '\000', sin6_family = 0 '\000', 
        sin6_port = 0, sin6_flowinfo = 0, sin6_addr = {__u6_addr = {
            __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 
0, 0, 0,
              0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id 
= 0}}},
  inp_list = {cle_next = 0xfffff80031d4e000, cle_prev = 
0xfffffe0048566828},
  inp_epoch_ctx = {data = {0xffffffff80759b10 <in_pcbfree_deferred>, 
      0xfffff80002334d08}}}
(kgdb) up
#10 0xffffffff80806464 in udp_input (mp=<optimized out>, 
offp=<optimized out>,
    proto=17) at /opt/src/svn-current/sys/netinet/udp_usrreq.c:722
722		if (udp_append(inp, ip, m, iphlen, udp_in) == 0) 
(kgdb) l
717				return (IPPROTO_DONE);
718			}
719		}
720	
721		UDP_PROBE(receive, NULL, inp, ip, inp, uh);
722		if (udp_append(inp, ip, m, iphlen, udp_in) == 0) 
723			INP_RUNLOCK(inp);
724		return (IPPROTO_DONE);
725	
726	badunlocked:
(kgdb) up 
#11 0xffffffff8075f645 in ip_input (m=0x0)
    at /opt/src/svn-current/sys/netinet/ip_input.c:825
825		(*inetsw[ip_protox[ip->ip_p]].pr_input)(&m, &hlen, ip->ip_p);
(kgdb) l
820		/*
821		 * Switch out to protocol's input routine.
822		 */
823		IPSTAT_INC(ips_delivered);
824	
825		(*inetsw[ip_protox[ip->ip_p]].pr_input)(&m, &hlen, ip->ip_p);
826		return;
827	bad:
828		m_freem(m);
829	}
(kgdb) 


In both cases rtsold was starting.



-- 
Cheers,
Cy Schubert <Cy.Schubert_at_cschubert.com>
FreeBSD UNIX:  <cy_at_FreeBSD.org>   Web:  http://www.FreeBSD.org

	The need of the many outweighs the greed of the few.
Received on Thu Jun 21 2018 - 23:15:05 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:16 UTC