Re: How to enable tcp bbr in FreeBSD???

> On 26. Apr 2020, at 15:09, Randall Stewart <rrs_at_netflix.com> wrote:
> 
> I am thinking that this really needs to have
> a deeper support in the transport.
> 
> I see that OOB is even in socket_dgram process and there
> are comments in there that indicate it may be a problem.. I 
> know UDP does not support it.
> 
> So I think what is needed here is
> 
> 1) a new pru_method that at send/rcv it can query if PRUS_EOF <or> PRUS_OOB
>   is supported.
Not sure what you need to check for PRUS_EOF?
> 2) For UDP either query would return false.
> 3) For TCP this would resolve to a stack specific query. For the 
>   freebsd stack, it would return true for both, for Rack or BBR
>   you would get True for PRUS_EOF and False for PRUS_OOB. This way
>   you could capture an error in any case by adding a check at the
>   top of send/recv and immediately return an error as appropriate.
> 4) I would also think like all pru methods, you get a default of true/true
>   so that way I guess unix domain sockets would continue as they are (not
>   sure if they support these or not I should probably look)
Hmm. Thinking about this:

I guess we want to focus on TCP, since my understanding is that the
problem is that some TCP stacks do support OOB, some don't. So you
can't query that right now via a TCP value.

So couldn't we check in tcp_usr_send() if the stack currently being
used for the socket support OOB? It would be a stack specific value.
Such a change should fix this issue and does not impact other protocols.

Best regards
Michael
> 
> 
> R
> 
>> On Apr 26, 2020, at 8:55 AM, Randall Stewart <rrs_at_netflix.com> wrote:
>> 
>> Sure..
>> 
>> I will take a look at it. 
>> 
>> R
>> 
>>> On Apr 26, 2020, at 8:51 AM, Michael Tuexen <Michael.Tuexen_at_macmic.franken.de> wrote:
>>> 
>>> 
>>> 
>>>> On 26. Apr 2020, at 13:34, Randall Stewart via freebsd-transport <freebsd-transport_at_freebsd.org> wrote:
>>>> 
>>>> I have pulled down the reproducers.. one thing to note
>>>> is they are all rack (though it could be that the problems
>>>> are also in BBR). And of course FreeBSD is behind NF in
>>>> rack at least.
>>>> 
>>>> I need to work on getting things updated.. one thing Michael, 
>>>> both Rack and BBR in NF have lost the OOB handling. Please do
>>>> not commit any more changes to Rack .. since that work has already
>>>> been done.
>>> I understand that the support of MSG_OOB is gone, but if you want
>>> to return an error to the user when he uses MSG_OOB, you need to
>>> trigger this error in the protocol specific code.
>>> 
>>> I don't think we can return an error in all cases (also for the
>>> default stack), since that would change existing behaviour.
>>> 
>>> I leave this up to you.
>>> 
>>> Best regards
>>> Michael
>>>> 
>>>> R
>>>> 
>>>>> On Apr 26, 2020, at 7:28 AM, Randall Stewart <rrs_at_netflix.com> wrote:
>>>>> 
>>>>> This is actually the first I have heard of these bugs…
>>>>> 
>>>>> I will have look at them Mark.
>>>>> 
>>>>> R
>>>>> 
>>>>>> On Apr 24, 2020, at 10:23 AM, Mark Johnston <markj_at_freebsd.org> wrote:
>>>>>> 
>>>>>> On Fri, Apr 24, 2020 at 03:15:08PM +0100, Tom Jones wrote:
>>>>>>> rrs_at_freebsd.org
>>>>>>> Bcc: 
>>>>>>> Subject: Re: How to enable tcp bbr in FreeBSD???
>>>>>>> Reply-To: 
>>>>>>> In-Reply-To: <6042155a-297b-d85e-1d64-24d93da329a2_at_gmail.com>
>>>>>>> 
>>>>>>> 
>>>>>>> ... snip ...
>>>>>>>> 
>>>>>>>> Maybe it is not ready for prime time, i do not know why it is not in the
>>>>>>>> default build.
>>>>>>>> Maybe ask the committer.
>>>>>>>> 
>>>>>>> 
>>>>>>> I have added rrs_at_ in cc and the freebsd-transport list. 
>>>>>>> 
>>>>>>> Does anyone know if there are plans to enable alternate TCP stacks in
>>>>>>> generic? 
>>>>>>> 
>>>>>>> Is there a stability point we need to hit first?
>>>>>> 
>>>>>> There are a couple of open bugs found by syzkaller (complete with
>>>>>> reproducers) that appeared when I enabled the alternate TCP stacks:
>>>>>> 
>>>>>> https://syzkaller.appspot.com/bug?id=986b4cecd84439df9794bda1a45d9cf0f50356fe
>>>>>> https://syzkaller.appspot.com/bug?id=048f650e99696f881872a285cef0e3b9bd4f4e25
>>>>>> 
>>>>>> I'd expect these to be fixed before providing the alternate stacks in
>>>>>> GENERIC.
>>>>>> _______________________________________________
>>>>>> freebsd-transport_at_freebsd.org mailing list
>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-transport
>>>>>> To unsubscribe, send any mail to "freebsd-transport-unsubscribe_at_freebsd.org"
>>>>> 
>>>>> ------
>>>>> Randall Stewart
>>>>> rrs_at_netflix.com
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> ------
>>>> Randall Stewart
>>>> rrs_at_netflix.com
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> freebsd-transport_at_freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-transport
>>>> To unsubscribe, send any mail to "freebsd-transport-unsubscribe_at_freebsd.org"
>> 
>> ------
>> Randall Stewart
>> rrs_at_netflix.com
>> 
>> 
>> 
> 
> ------
> Randall Stewart
> rrs_at_netflix.com
> 
> 
>