Re: LORs with PF

On Thursday 22 July 2004 14:36, Robert Watson wrote:
> On Thu, 22 Jul 2004, Bjoern A. Zeeb wrote:
> > > I don't think, it was reported yet, but here it goes:
> > >
> > > lock order reversal
> > >  1st 0xc0632c80 pf task mtx (pf task mtx) _at_
> > > /usr/src/sys/contrib/pf/net/pf.c:5822 2nd 0xc066638c tcp (tcp) _at_
> > > /usr/src/sys/contrib/pf/net/pf.c:2420
> >
> > for reference: added it to "LOR page" with LOR # 017:
> > http://sources.zabbadoz.net/freebsd/lor.html#017
>
> I have no immediate fix, but here's an explanation:
>
> This likely occurs because calls are made into pf in the outgoing packet
> path, which may hold protocol layer locks (inpcb and pcbinfo locks).  The
> pf mutex will therefore typically follow the protocol locks in the lock
> order.  However, when pf calls up to the pcb/socket code to find
> potentially matching pcbs or sockets for a packet, it can't/shouldn't hold
> the pf lock across that.  A similar bug exists in ipfw.
>
> FWIW, this is actually symptomatic of a race: technically, that pcb lookup
> could have a different result between when the firewall looks it up and
> when the TCP or UDP code looks it up.  It's a property of what is
> generally considered a layering violation, albeit a useful one.
>
> The solution I'm looking at for ipfw, but haven't had a chance to
> prototype yet, is to avoid holding the mutex for the duration of the call
> to ipfw, but instead use some rule set reference count such that ipfw
> consumers hold a reference on the rules to prevent them from changing out
> from under them.  Note that this has interesting semantic consideration
> such as starvation, which are usually remedied using copy-on-write, but
> that may be expensive for large rule sets.  Just something to think
> about...

This raises the question (at least for me) for a useable sx(9) implementation 
or a busycount of some sort. Somebody suggested atomic refcounts, but I don't 
remember what the outcome was. I know of at least two places where we have 
handrolled busycounts: pfil_hooks and MAC
Something general with tuneable starvation scheme might be worthwhile?

In any case, I must look into something of that kind for pf as well. At the 
moment we have a lot of: "drop lock - call back into the stack - reaquire 
lock".

-- 
/"\  Best regards,			| mlaier_at_freebsd.org
\ /  Max Laier				| ICQ #67774661
 X   http://pf4freebsd.love2party.net/	| mlaier_at_EFnet
/ \  ASCII Ribbon Campaign		| Against HTML Mail and News