Re: pf crash on -current

On 2015-02-24 08:05:47 (+0100), Kristof Provost <kristof_at_sigsegv.be> wrote:
> On 2015-02-23 17:23:55 (-0800), Davide Italiano <davide_at_freebsd.org> wrote:
> > The bt you posted suggest this could be stack overflow, probably due
> > to infinite recursion.
> > Also, as a wild guess, just looking at the stacktrace, I think this
> > might be related to the recent ipv6 fragment changes. Try to back them
> > out, and see if things gets more stable ( r278831 and r278843).
> > 
> That's almost certainly what it is.
> 
After a bit of fiddling around I've managed to reproduce this locally.

Essentially we get caught in a loop of defragmenting and refragmenting:
Fragmented packets come in on one interface and get collected until we
can defragment it. At that point the defragmented packet is handed back
to the ip stack (at the pfil point in ip6_input(). Normal processing
continues.
Eventually we figure out that the packet has to be forwarded and we end
up at the pfil hook in ip6_forward(). After doing the inspection on the
defragmented packet we see that the packet has been defragmented and
because we're forwarding we have to refragment it. That's indicated by
the presence of the PF_REASSEMBLED tag.

In pf_refragment6() we remove that tag, split the packet up again and
then ip6_forward() the individual fragments.
Those fragments hit the pfil hook on the way out, so they're
collected until we can reconstruct the full packet, at which point we're
right back where we left off and things continue until we run out of
stack.

There are two reasons Allan is seeing this and no one else has so far.

The first is that he's scrubbing both on input and output. My own tests
have always been done with 'scrub in all fragment reassemble', rather
than 'scrub all fragment reassemble' so I didn't see this problem.

The second is that he's got an internal interface with a higher MTU,
so the refragmentation actually works for him.
There's an open problem where ip6_forward() drops the defragmented
packet before the pfil(PFIL_OUT) hook because it's too big for the
output interface.
If the last patch of my series (https://reviews.freebsd.org/D1815) had
been merged as well more people would have been affected.

One possible fix for Allan's problem would be to tag the fragments after
refragmentation so that pf ignores them. After all, the defragmented
packet has already been inspected so there's no point in checking the
fragments again.

I have the feeling there's a way to fix this problem and the issue D1815
tries to fix in one go though. I'll need to think about it a bit more.

Regards,
Kristof