Re: ng_snd_item: Panic?

On 06/25/2019 3:13 pm, Andrey V. Elsukov wrote:
> On 25.06.2019 15:59, Larry Rosenman wrote:
>> On 06/25/2019 4:18 am, Andrey V. Elsukov wrote:
>>> On 24.06.2019 23:10, Larry Rosenman wrote:
>>>>>> #5  0xffffffff828ee5b7 in ng_snd_item (item=0xfffff8021e3b4d80,
>>>>>> flags=0)
>>>>>>     at /usr/src/sys/netgraph/ng_base.c:2252
>>>>> 
>>>>> It looks like you use some netgraph based ethernet interface.
>>>>> The system got received ARP request and is going to send the reply,
>>>>> but somehow mbuf with this ARP request has initialized m_next 
>>>>> pointer,
>>>>> thus it is considered as a chain of mbufs.
>>>>> 
>>>>> in_arpinput() reuses received mbuf to construct the reply, but it
>>>>> doesn't check that an mbut is a chain. It just sets m_len and sends 
>>>>> it.
>>>>> Then since you have INVARIANTS in your kernel, the netgraph code 
>>>>> check
>>>>> the actual length of the chain, and it doesn't match to m_len. It
>>>>> panics.
>>>> 
>>>> 
>>>> so, is this a bug?  Timing race? Other?
>>> 
>>> I think we should determine that my assumption is correct :)
>>> Can you show the output of the following commands from the kgdb for 
>>> this
>>> core?
>>> 
>>> (kgdb) f 7
>>> (kgdb) p *m
>>> (kgdb) p *m->m_next
>> 
>> 
>> (kgdb) fr 7
>> #7  0xffffffff805b1e43 in ether_output (ifp=<optimized out>,
>> m=0xfffff81f59eefb00, dst=0xfffffe012628d740, ro=<optimized out>) at
>> /usr/src/sys/net/if_ethersubr.c:430
>> 430            if ((error = (*ng_ether_output_p)(ifp, &m)) != 0) {
> 
> I failed to track the possible way to get this.
> Please, show the output of the following commands:
> (kgdb) f 7
> (kgdb) p/x (u_char[42])m->m_data
> (kgdb) p/x (u_char[1372]m->m_next->m_data
> 
> Did you used this configuration for the long time and these panics were
> the first time?


(kgdb) f 7
#7  0xffffffff805b1e43 in ether_output (ifp=<optimized out>,
     m=0xfffff81f59eefb00, dst=0xfffffe012628d740, ro=<optimized out>)
     at /usr/src/sys/net/if_ethersubr.c:430
430			if ((error = (*ng_ether_output_p)(ifp, &m)) != 0) {
(kgdb) p/x (u_char[42])m->m_data
$1 = {0x72, 0xfb, 0xee, 0x59, 0x1f, 0xf8, 0xff, 0xff, 0x2a, 0x0, 0x0, 
0x0,
   0x1, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x60, 0xd, 
0x53,
   0x4c, 0x10, 0xf8, 0xff, 0xff, 0x2a, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 
0x0,
   0x0}
(kgdb) p/x (u_char[1372]m->m_next->m_data
A syntax error in expression, near `m->m_next->m_data'.
(kgdb) p/x (u_char[1372])m->m_next->m_data
$2 = {0x0, 0xb8, 0x44, 0x21, 0x0, 0xf8, 0xff, 0xff, 0x5c, 0x5, 0x0, 0x0, 
0x1,
   0x1, 0x0 <repeats 58 times>, 0x1, 0x0, 0x0, 0x0, 0xde, 0xc0, 0xad, 
0xde,
   0x0, 0xb8, 0x44, 0x21, 0x0, 0xf8, 0xff, 0xff, 0x0, 0x8, 0x0, 0x0, 0x6, 
0x1,
   0x0 <repeats 26 times>, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 0xde, 0xde, 0xc0, 0xad, 
0xde,
   0xde, 0xc0, 0xad, 0xde, 0x0 <repeats 16 times>, 0x4c, 0xe0, 0xd6, 
0x79,
   0x19, 0xf8, 0xff, 0xff, 0xa8, 0x3, 0x0, 0x0, 0x1, 0x3...}
(kgdb)


I've been using this configuration for quite a few months, and these 2 
panics yesterday are the only ones
I've seen.


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106