Re: sr-iov issues, reset_hw() failed with error -100

From: Steven Hartland <killing_at_multiplay.co.uk>
Date: Mon, 22 Feb 2016 21:35:57 +0000
isn't that 48 cores (12 real 12 virtual) per CPU?

On 22/02/2016 21:26, Ultima wrote:
> This system has 24 cores (e5-2670v3)x2
>
> Ultima
>
> On Mon, Feb 22, 2016 at 3:53 PM, Pieper, Jeffrey E <
> jeffrey.e.pieper_at_intel.com> wrote:
>
>> Just out of curiosity, how many cores does your system have?
>>
>> Jeff
>>
>> -----Original Message-----
>> From: owner-freebsd-current_at_freebsd.org [mailto:
>> owner-freebsd-current_at_freebsd.org] On Behalf Of Ultima
>> Sent: Monday, February 22, 2016 12:02 PM
>> To: Eric Joyner <erj_at_freebsd.org>
>> Cc: freebsd-current_at_freebsd.org; freebsd-virtualization_at_freebsd.org
>> Subject: Re: sr-iov issues, reset_hw() failed with error -100
>>
>> After reboot...
>>
>> ifconfig ix1 up
>>
>> dhclient ix1
>> DHCPDISCOVER on ix1 to 255.255.255.255 port 67 interval 4
>> DHCPOFFER from 192.168.1.1
>> DHCPREQUEST on ix1 to 255.255.255.255 port 67
>> DHCPACK from 192.168.1.1
>> bound to 192.168.1.145 -- renewal in 21600 seconds.
>>
>> ix0 down
>> ping 192.168.1.1
>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>> 64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.149 ms
>> 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.171 ms
>> 64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.167 ms
>>
>> iovctl -Cf /etc/iovctl.conf
>>
>> ping 192.168.1.1
>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>> ^C
>> --- 192.168.1.1 ping statistics ---
>> 29 packets transmitted, 0 packets received, 100.0% packet loss
>> ifconfig ix1 up
>> ping 192.168.1.1
>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>> ^C
>> --- 192.168.1.1 ping statistics ---
>> 12 packets transmitted, 0 packets received, 100.0% packet loss
>>
>> ix1 is no longer usable until a restart...
>>
>> iovctl -Dd ix1
>> ifconfig ix1 up
>> ping 192.168.1.1
>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>> ^C
>> --- 192.168.1.1 ping statistics ---
>> 9 packets transmitted, 0 packets received, 100.0% packet loss
>>
>>
>>
>> Is there anything else that maybe useful?
>>
>> here is my ifconfig at the end (after ifconfig ix0 up)
>>
>>
>> ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
>> mtu 1500
>>
>> options=e400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
>> ether -Hidden-
>> inet 192.168.1.8 netmask 0xffffff00 broadcast 192.168.1.255
>> inet 192.168.1.9 netmask 0xffffff00 broadcast 192.168.1.255
>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>> media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
>> status: active
>> ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>
>> options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
>> ether -Hidden-
>> inet 192.168.1.145 netmask 0xffffff00 broadcast 192.168.1.255
>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>> media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
>> status: active
>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>> options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>> inet6 ::1 prefixlen 128
>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
>> inet 127.0.0.1 netmask 0xff000000
>> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>> groups: lo
>> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
>> 1500
>> ether -Hidden-
>> nd6 options=9<PERFORMNUD,IFDISABLED>
>> groups: bridge
>> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>> member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>         ifmaxaddr 0 port 1 priority 128 path cost 2000
>> member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>         ifmaxaddr 0 port 5 priority 128 path cost 2000
>> epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
>> 0 mtu 1500
>> options=8<VLAN_MTU>
>> ether -Hidden-
>> inet6 fe80::ff:70ff:fe00:50a%epair0a prefixlen 64 scopeid 0x5
>> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
>> status: active
>> groups: epair
>>
>> On Mon, Feb 22, 2016 at 1:51 PM, Eric Joyner <erj_at_freebsd.org> wrote:
>>
>>> Did you do an ifconfig up on ix1 before loading the VF driver?
>>>
>>> On Sat, Feb 20, 2016 at 11:57 AM Ultima <ultima1252_at_gmail.com> wrote:
>>>
>>>>   Decided to do some testing with iovctl to see how sr-iov is coming
>> along.
>>>> Currently when adding the vf's there are a couple errors, and the
>> network
>>>> no longer function after iovctl is started. My guess is the reset_hw()
>>>> call
>>>> that is failing. Any ideas why this call would fail? I tested this on
>> both
>>>> ports, ix1 is detached and unused for this test, however inserting a
>> cable
>>>> results in an unusable port. iovctl -Dd ix1 removes the vf's, however
>>>> functionality is still not restored without a system restart.
>>>>
>>>> FreeBSD S1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r295736: Wed Feb 17
>>>> 21:17:28 EST 2016     root_at_S1:/usr/obj/usr/src/sys/MYKERNEL  amd64
>>>>
>>>> /boot/loader.conf
>>>> hw.ix.num_queues="4"
>>>>
>>>> /etc/iovctl.conf
>>>> PF {
>>>>          device : ix1;
>>>>          num_vfs : 31;
>>>> }
>>>>
>>>> DEFAULT {
>>>>          passthrough : true;
>>>> }
>>>> VF-0 {
>>>>          passthrough : false;
>>>> }
>>>> VF-1 {
>>>>          passthrough : false;
>>>> }
>>>>
>>>> # iovctl -C -f /etc/iovctl.conf
>>>>
>>>> dmesg
>>>> ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver, Version -
>>>> 1.4.6-k> at device 0.129 on pci12
>>>> ixv0: Using MSIX interrupts with 2 vectors
>>>> ixv0: ixgbe_reset_hw() failed with error -100
>>>> device_attach: ixv0 attach returned 5
>>>> ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver, Version -
>>>> 1.4.6-k> at device 0.131 on pci12
>>>> ixv0: Using MSIX interrupts with 2 vectors
>>>> ixv0: ixgbe_reset_hw() failed with error -100
>>>> device_attach: ixv0 attach returned 5
>>>> pci12: <network, ethernet> at device 0.133 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.135 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.137 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.139 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.141 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.143 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.145 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.147 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.149 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.151 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.153 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.155 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.157 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.159 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.161 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.163 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.165 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.167 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.169 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.171 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.173 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.175 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.177 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.179 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.181 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.183 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.185 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.187 (no driver attached)
>>>> pci12: <network, ethernet> at device 0.189 (no driver attached)
>>>>
>>>> pciconf -lv
>>>> ix1_at_pci0:129:0:1:       class=0x020000 card=0x00001458 chip=0x15288086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'Ethernet Controller 10-Gigabit X540-AT2'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> none155_at_pci0:129:0:129: class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> none156_at_pci0:129:0:131: class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt0_at_pci0:129:0:133:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt1_at_pci0:129:0:135:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt2_at_pci0:129:0:137:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt3_at_pci0:129:0:139:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt4_at_pci0:129:0:141:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt5_at_pci0:129:0:143:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt6_at_pci0:129:0:145:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt7_at_pci0:129:0:147:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt8_at_pci0:129:0:149:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt9_at_pci0:129:0:151:    class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt10_at_pci0:129:0:153:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt11_at_pci0:129:0:155:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt12_at_pci0:129:0:157:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt13_at_pci0:129:0:159:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt14_at_pci0:129:0:161:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt15_at_pci0:129:0:163:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt16_at_pci0:129:0:165:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt17_at_pci0:129:0:167:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt18_at_pci0:129:0:169:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt19_at_pci0:129:0:171:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt20_at_pci0:129:0:173:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt21_at_pci0:129:0:175:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt22_at_pci0:129:0:177:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt23_at_pci0:129:0:179:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt24_at_pci0:129:0:181:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt25_at_pci0:129:0:183:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt26_at_pci0:129:0:185:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt27_at_pci0:129:0:187:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>> ppt28_at_pci0:129:0:189:   class=0x020000 card=0x00001458 chip=0x15158086
>>>> rev=0x01 hdr=0x00
>>>>      vendor     = 'Intel Corporation'
>>>>      device     = 'X540 Ethernet Controller Virtual Function'
>>>>      class      = network
>>>>      subclass   = ethernet
>>>>
>>>> Ultima
>>>> _______________________________________________
>>>> freebsd-current_at_freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>>> To unsubscribe, send any mail to "
>> freebsd-current-unsubscribe_at_freebsd.org
>>>> "
>>>>
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>>
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
Received on Mon Feb 22 2016 - 20:35:57 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:03 UTC