Reproducible igb related panic 11.0-ALPHA4 OS: FreeBSD 11.0-ALPHA4 #6 r302022 Hardware: Asus P9D C224 (integrated <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k) Hi, Kernel panics within a few seconds with heavy network load. Using "iperf -c another_host" is sufficient to induce the problem. Setting hw.igb.enable_msix=0 in loader.conf "solves" the problem. Below is the first few pages of crashinfo, + dmesg. The problem occurs on 2 instances of similar hardware (with same motherboards). I have full coredump available and am happy to assist with debugging/testing. Cheers, --Richard sandbox.ci.com.au dumped core - see /u0/crash/vmcore.1 Mon Jun 20 14:25:56 AEST 2016 FreeBSD sandbox.ci.com.au 11.0-ALPHA4 FreeBSD 11.0-ALPHA4 #6 r302022: Mon Jun 20 09:27:41 AEST 2016 root_at_sandbox.ci.com.au:/u0/obj/usr/src/sys/LOCAL amd64 panic: sbsndptr: sockbuf 0xfffff800a3e3db58 and mbuf 0xfffff8004995ae00 clashing GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: sbsndptr: sockbuf 0xfffff800a3e3db58 and mbuf 0xfffff8004995ae00 clashing cpuid = 2 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe07c6777210 vpanic() at vpanic+0x182/frame 0xfffffe07c6777290 panic() at panic+0x43/frame 0xfffffe07c67772f0 sbsndptr() at sbsndptr+0xda/frame 0xfffffe07c6777300 tcp_output() at tcp_output+0xf45/frame 0xfffffe07c67774b0 tcp_do_segment() at tcp_do_segment+0x2d01/frame 0xfffffe07c67775c0 tcp_input() at tcp_input+0xd1c/frame 0xfffffe07c6777720 ip_input() at ip_input+0x15f/frame 0xfffffe07c6777780 netisr_dispatch_src() at netisr_dispatch_src+0xa5/frame 0xfffffe07c67777e0 ether_demux() at ether_demux+0x12a/frame 0xfffffe07c6777810 ether_nh_input() at ether_nh_input+0x322/frame 0xfffffe07c6777870 netisr_dispatch_src() at netisr_dispatch_src+0xa5/frame 0xfffffe07c67778d0 ether_input() at ether_input+0x26/frame 0xfffffe07c67778f0 igb_rxeof() at igb_rxeof+0x81c/frame 0xfffffe07c6777990 igb_msix_que() at igb_msix_que+0x152/frame 0xfffffe07c67779e0 intr_event_execute_handlers() at intr_event_execute_handlers+0x20f/frame 0xfffffe07c6777a20 ithread_loop() at ithread_loop+0xc6/frame 0xfffffe07c6777a70 fork_exit() at fork_exit+0x85/frame 0xfffffe07c6777ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe07c6777ab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done. done. Loaded symbols for /boot/kernel/uhid.ko Reading symbols from /boot/kernel/imgact_binmisc.ko...Reading symbols from /usr/lib/debug//boot/kernel/imgact_binmisc.ko.debug...done. done. Loaded symbols for /boot/kernel/imgact_binmisc.ko #0 doadump (textdump=0) at pcpu.h:221 221 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=0) at pcpu.h:221 #1 0xffffffff8039ee7b in db_dump (dummy=<value optimized out>, dummy2=false, dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533 #2 0xffffffff8039ec79 in db_command (cmd_table=<value optimized out>) at /usr/src/sys/ddb/db_command.c:440 #3 0xffffffff8039e9d4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493 #4 0xffffffff803a19bb in db_trap (type=<value optimized out>, code=<value optimized out>) at /usr/src/sys/ddb/db_main.c:251 #5 0xffffffff80b3e3d3 in kdb_trap (type=<value optimized out>, code=<value optimized out>, tf=<value optimized out>) at /usr/src/sys/kern/subr_kdb.c:654 #6 0xffffffff8100fa95 in trap (frame=0xfffffe07c6777140) at /usr/src/sys/amd64/amd64/trap.c:556 #7 0xffffffff80ff2421 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0xffffffff80b3da7b in kdb_enter (why=0xffffffff81533552 "panic", msg=0xffffffff81a960f8 " \035\027\201ÿÿÿÿ") at cpufunc.h:63 #9 0xffffffff80af52bf in vpanic (fmt=<value optimized out>, ap=0xfffffe07c67772d0) at /usr/src/sys/kern/kern_shutdown.c:752 #10 0xffffffff80af5113 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:690 #11 0xffffffff80b8f78a in sbsndptr (sb=<value optimized out>, off=<value optimized out>, len=<value optimized out>, moff=<value optimized out>) at /usr/src/sys/kern/uipc_sockbuf.c:1196 #12 0xffffffff80d27fb5 in tcp_output (tp=<value optimized out>) at /usr/src/sys/netinet/tcp_output.c:1045 #13 0xffffffff80d24f01 in tcp_do_segment (m=<value optimized out>, th=<value optimized out>, so=0xfffff800a3e3d9a8, tp=<value optimized out>, drop_hdrlen=52, tlen=<value optimized out>, iptos=<value optimized out>, ti_locked=Cannot access memory at address 0x1 ) at /usr/src/sys/netinet/tcp_input.c:3161 #14 0xffffffff80d2152c in tcp_input (mp=<value optimized out>, offp=<value optimized out>, proto=<value optimized out>) at /usr/src/sys/netinet/tcp_input.c:1442 #15 0xffffffff80c929cf in ip_input (m=Cannot access memory at address 0x0 ) at /usr/src/sys/netinet/ip_input.c:798 #16 0xffffffff80c24505 in netisr_dispatch_src (proto=1, source=<value optimized out>, m=0x12) at /usr/src/sys/net/netisr.c:1121 #17 0xffffffff80c0bb0a in ether_demux (ifp=<value optimized out>, m=0xffffffff81a960f8) at /usr/src/sys/net/if_ethersubr.c:850 #18 0xffffffff80c0c762 in ether_nh_input (m=<value optimized out>) at /usr/src/sys/net/if_ethersubr.c:639 #19 0xffffffff80c24505 in netisr_dispatch_src (proto=5, source=<value optimized out>, m=0x12) at /usr/src/sys/net/netisr.c:1121 #20 0xffffffff80c0bd86 in ether_input (ifp=<value optimized out>, m=0x0) at /usr/src/sys/net/if_ethersubr.c:759 #21 0xffffffff80564d1c in igb_rxeof (count=-1543849472) at /usr/src/sys/dev/e1000/if_igb.c:4956 #22 0xffffffff80564072 in igb_msix_que (arg=0xfffff800096e50d0) at /usr/src/sys/dev/e1000/if_igb.c:1611 #23 0xffffffff80aadcff in intr_event_execute_handlers ( p=<value optimized out>, ie=<value optimized out>) at /usr/src/sys/kern/kern_intr.c:1262 #24 0xffffffff80aae316 in ithread_loop (arg=<value optimized out>) at /usr/src/sys/kern/kern_intr.c:1275 #25 0xffffffff80aaa945 in fork_exit ( callout=0xffffffff80aae250 <ithread_loop>, arg=0xfffff80006e80040, frame=0xfffffe07c6777ac0) at /usr/src/sys/kern/kern_fork.c:1038 #26 0xffffffff80ff295e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #27 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) dmesg Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-ALPHA3 #5 r301877: Tue Jun 14 15:48:23 AEST 2016 root_at_fmaster.ci.com.au:/u0/obj/usr/src/sys/LOCAL amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) can't re-use a leaf (ixl_rx_miss_bufs)! VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E3-1271 v3 _at_ 3.60GHz (3591.75-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x21<LAHF,ABM> Structured Extended Features=0x2fbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG> XSAVE Features=0x1<XSAVEOPT> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33250164736 (31709 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <ALASKA A M I> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads random: unblocking device. ioapic0 <Version 2.0> irqs 0-23 on motherboard random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff81076900, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" vtvga0: <VT VGA driver> on motherboard cryptosoft0: <software crypto> on motherboard acpi0: <ALASKA A M I> on motherboard acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 cpu4: <ACPI CPU> on acpi0 cpu5: <ACPI CPU> on acpi0 cpu6: <ACPI CPU> on acpi0 cpu7: <ACPI CPU> on acpi0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. Event timer "RTC" frequency 32768 Hz quality 0 attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xdc300000-0xdc30ffff irq 16 at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA xhci0: Port routing mask set to 0xffffffff usbus0 on xhci0 ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xdc314000-0xdc3143ff irq 20 at device 26.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci0 pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0 pci1: <ACPI PCI bus> on pcib1 igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xe000-0xe01f mem 0xdc200000-0xdc27ffff,0xdc280000-0xdc283fff irq 16 at device 0.0 on pci1 igb0: Using MSIX interrupts with 5 vectors igb0: Ethernet address: 14:dd:a9:4d:c7:b8 igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb0: netmap queues/slots: TX 4/1024, RX 4/1024 pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0 pci2: <ACPI PCI bus> on pcib2 igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xd000-0xd01f mem 0xdc100000-0xdc17ffff,0xdc180000-0xdc183fff irq 17 at device 0.0 on pci2 igb1: Using MSIX interrupts with 5 vectors igb1: Ethernet address: 14:dd:a9:4d:c7:b9 igb1: Bound queue 0 to cpu 4 igb1: Bound queue 1 to cpu 5 igb1: Bound queue 2 to cpu 6 igb1: Bound queue 3 to cpu 7 igb1: netmap queues/slots: TX 4/1024, RX 4/1024 pcib3: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0 pci3: <ACPI PCI bus> on pcib3 pcib4: <ACPI PCI-PCI bridge> irq 18 at device 0.0 on pci3 pci4: <ACPI PCI bus> on pcib4 vgapci0: <VGA-compatible display> port 0xc000-0xc07f mem 0xd8000000-0xdbffffff,0xdc000000-0xdc01ffff irq 18 at device 0.0 on pci4 vgapci0: Boot video device pcib5: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0 pci5: <ACPI PCI bus> on pcib5 pcib6: <ACPI PCI-PCI bridge> irq 19 at device 0.0 on pci5 pci6: <ACPI PCI bus> on pcib6 ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xdc313000-0xdc3133ff irq 23 at device 29.0 on pci0 usbus2: EHCI version 1.0 usbus2 on ehci1 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf020-0xf03f mem 0xdc312000-0xdc3127ff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 ahciem0: <AHCI enclosure management bridge> on ahci0 acpi_button0: <Power Button> on acpi0 acpi_tz0: <Thermal Zone> on acpi0 acpi_tz1: <Thermal Zone> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: cannot reserve I/O port range est0: <Enhanced SpeedStep Frequency Control> on cpu0 est1: <Enhanced SpeedStep Frequency Control> on cpu1 est2: <Enhanced SpeedStep Frequency Control> on cpu2 est3: <Enhanced SpeedStep Frequency Control> on cpu3 est4: <Enhanced SpeedStep Frequency Control> on cpu4 est5: <Enhanced SpeedStep Frequency Control> on cpu5 est6: <Enhanced SpeedStep Frequency Control> on cpu6 est7: <Enhanced SpeedStep Frequency Control> on cpu7 Timecounters tick every 1.000 msec nvme cam probe device init usbus0: 5.0Gbps Super Speed USB v3.0 usbus1: 480Mbps High Speed USB v2.0 ugen0.1: <0x8086> at usbus0 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ugen1.1: <Intel> at usbus1 uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 usbus2: 480Mbps High Speed USB v2.0 ugen2.1: <Intel> at usbus2 uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: <ST9500420AS 0005SDM1> ATA8-ACS SATA 2.x device ada0: Serial Number 5VJ7J98V ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors) ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: <INTEL SSDSC2BW120A4 DC32> ACS-2 ATA SATA 3.x device ada1: Serial Number PHDA4084001E1207GN ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 114473MB (234441648 512 byte sectors) ada1: quirks=0x1<4K> ses0 at ahciem0 bus 0 scbus6 target 0 lun 0 ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device uhub1: 2 ports with 2 removable, self powered ses0: SEMB SES Device SMP: AP CPU #1 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #5 Launched! Timecounter "TSC-low" frequency 1795876737 Hz quality 1000 Trying to mount root from ufs:/dev/gpt/ssdroot [rw]... uhub2: 2 ports with 2 removable, self powered WARNING: / was not properly dismounted sysctl: oid 'debug.ddb.capture.maxbufsize' is read only at line 13 Setting hostuuid: 709109c0-5bcb-11d9-8a6e-54a05085d521. Setting hostid: 0xcd1c5563. Starting ddb. eval: limits: not found /etc/rc: WARNING: failed to start ddb Starting file system checks: uhub0: 21 ports with 21 removable, self powered ugen1.2: <vendor 0x8087> at usbus1 uhub3: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.05, addr 2> on usbus1 ugen2.2: <vendor 0x8087> at usbus2 uhub4: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.05, addr 2> on usbus2 uhub3: 6 ports with 6 removable, self powered /dev/gpt/ssdroot: 157812 files, 310588 used, 197195 free (139 frags, 24632 blocks, 0.0% fragmentation) /dev/gpt/ssdvar: DEFER FOR BACKGROUND CHECKING /dev/gpt/ssdtmp: DEFER FOR BACKGROUND CHECKING /dev/gpt/ssdusr: DEFER FOR BACKGROUND CHECKING /dev/gpt/ssdhome: DEFER FOR BACKGROUND CHECKING /dev/gpt/ssdu0: DEFER FOR BACKGROUND CHECKING ugen0.2: <NOVATEK> at usbus0 ukbd0: <NOVATEK USB Keyboard, class 0/0, rev 1.10/1.12, addr 1> on usbus0 kbd2 at ukbd0Received on Mon Jun 20 2016 - 03:09:12 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:06 UTC