Re: (boost::)asio and kqueue problem

On Wed, Jul 20, 2016 at 12:38 PM, Konstantin Belousov
<kostikbel_at_gmail.com> wrote:
> On Tue, Jul 19, 2016 at 05:35:59PM +0200, Hartmut Brandt wrote:
>> Hi,
>>
>> I'm trying to use asio (that's boost::asio without boost) to handle
>> listening sockets asynchronuosly. This appears not to work. There are also
>> some reports on the net about this problem. I was able to reproduce the
>> problem with a small C-programm that does the same steps as asio. The
>> relevant sequence of system calls is:
>>
>> kqueue()                                       = 3 (0x3)
>> socket(PF_INET,SOCK_STREAM,6)                  = 4 (0x4)
>> setsockopt(0x4,0xffff,0x800,0x7fffffffea2c,0x4)        = 0 (0x0)
>> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0)
>> setsockopt(0x4,0xffff,0x4,0x7fffffffea2c,0x4)  = 0 (0x0)
>> bind(4,{ AF_INET 0.0.0.0:8080 },16)            = 0 (0x0)
>> listen(0x4,0x80)                               = 0 (0x0)
>> ioctl(4,FIONBIO,0xffffea2c)                    = 0 (0x0)
>> kevent(3,{ 4,EVFILT_READ,EV_ADD|EV_CLEAR,0x0,0x0,0x0 4,EVFILT_WRITE,EV_ADD|EV_CLEAR,0x0,0x0,0x0 },2,0x0,0,0x0) = 0 (0x0)
>> kevent(3,0x0,0,0x7fffffffe5a0,32,0x0)          ERR#4 'Interrupted system call'
>>
>> The problem here is that asio registers each file descriptor with
>> EVFILT_READ and EVFILT_WRITE as soon as it is opened (first kevent call).
>> After bringing the socket into the listening state and when async_accept()
>> is called it registers the socket a second time. According to the man page
>> this is perfectly legal and can be used to modify the registration.
>>
>> With this sequence of calls kevent() does not return when a connection is
>> established successfully.
>>
>> I tracked down the problem and the reason is in soo_kqfilter(). This is
>> called for the first EVFILT_READ registration and decides based on the
>> SO_ACCEPTCONN flag which filter operations to use solisten_filtops or
>> soread_filtops. In this case it chooses soread_filtops.
>>
>> The second EVFILT_READ registration does not call soo_kqfilter() again,
>> but just updates the filter from the data and fflags field so the
>> listening socket ends up with the wrong filter operations.
>>
>> The attached patch fixes this (kind of) by using the f_touch
>> operation (currently used only by the user filter). The filt_sotouch()
>> function changes the operation pointer in the knote when the socket is now
>> in the listening state. I suppose that the required locking is already
>> done in kqueue_register(), but I'm not sure. Asynchronous accepting now works.
>>
>> A better fix would probably be to change the operation vector on all
>> knotes attached to the socket in solisten(), but I fear I don't have the
>> necessary understanding of the locking that is required for this.
>>
>> Could somebody with enough kqueue() knowledge look whether the patch is
>> correct lock-wise?
> I find it weird that the fix still requires re-registration of the socket
> event to get it working after socket is marked as listen.  In other words,
> until the re-registration is done, the events for the registered filter
> are lost.
>
> IMO more correct solution would be to merge the filt_solisten and
> filt_soread, deciding which path to take by testing the SO_ACCEPTCON
> flag in the f_event() op.

This is reasonable.

Thanks,
sephe

-- 
Tomorrow Will Never Die