Re: [net] protecting interfaces from races between control and data ?

From: Adrian Chadd <adrian_at_freebsd.org>
Date: Mon, 5 Aug 2013 10:17:27 -0700
I'm travelling back to San Jose today; poke me tomorrow and I'll brain
dump what I did in ath(4) and the lessons learnt.

The TL;DR version - you don't want to grab an extra lock in the
read/write paths as that slows things down. Reuse the same per-queue
TX/RX lock and have:

* a reset flag that is set when something is resetting; that says to
the queue "don't bother processing anything, just dive out";
* 'i am doing Tx / Rx' flags per queue that is set at the start of
TX/RX servicing and finishes at the end; that way the reset code knows
if there's something pending;
* have the reset path grab each lock, set the 'reset' flag on each,
then walk each queue again and make sure they're all marked as 'not
doing TX/RX'. At that point the reset can occur, then the flag cna be
cleared, then TX/RX can resume.



-adrian

On 5 August 2013 10:13, Navdeep Parhar <np_at_freebsd.org> wrote:
> On 08/05/13 09:15, Luigi Rizzo wrote:
>> On Mon, Aug 5, 2013 at 5:46 PM, Adrian Chadd <adrian_at_freebsd.org> wrote:
>>
>>> On 5 August 2013 07:59, Bryan Venteicher <bryanv_at_daemoninthecloset.org>
>>> wrote:
>>>
>>>> What I've done in my drivers is:
>>>>   * Lock the core mutex
>>>>   * Clear IFF_DRV_RUNNING
>>>>   * Lock/unlock each queue's lock
>>>
>>> .. and I think that's the only sane way of doing it.
>>>
>>
>> yeah, this was also the solution we had in mind, i was surprised
>> not find this pattern in the drivers i have looked at.
>>
>> Also there are drivers (chelsio ?) which do not seem to have locks on the
>> receive interrupt handlers ?
>
> This is correct.  cxgbe(4) does not have any locks on rx, just a "state"
> for each rx queue that's maintained with atomic ops.
>
> Regards,
> Navdeep
>
>
>>
>> Does anyone know how linux copes with the same problem ?
>>
>> They seem to have an rtnl_lock() which is a global lock for all
>> configuration
>> of netdevices (would replace our per-interface 'core lock' above),
>> but i am totally unclear on how individual tx threads and interrupt handlers
>> acknowledge that they have read the change in status.
>>
>> cheers
>> luigi
>> _______________________________________________
>> freebsd-net_at_freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe_at_freebsd.org"
>>
>
Received on Mon Aug 05 2013 - 15:17:30 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:40 UTC