Re: [CFT] new sendfile(2)

On 17 February 2014 03:32, Gleb Smirnoff <glebius_at_freebsd.org> wrote:
> On Mon, Feb 17, 2014 at 11:24:21AM +0000, David Chisnall wrote:
> D> >  Now for the new sendfile. The core idea is that sendfile()
> D> > schedules the I/O, but doesn't wait for it to complete. It
> D> > returns immediately to the process, and I/O completion is
> D> > processed in kernel context. Unlike aio(4), no additional
> D> > threads in kernel are created. The new sendfile is a drop-in
> D> > replacement for the old one. Applications (like nginx) doesn't
> D> > need recompile, neither configuration change. The SF_NODISKIO is
> D> > ignored.
> D>
> D> Doesn't this introduce a race?  If I do a sendfile now, then I am at liberty to modify the underlying file as soon as it returns. With this version, I not only am not free to modify the file, I have no notification that it is finished so I can't ever safely use this call on a file that I might eventually modify.
> D>
> D> Wouldn't it be better to provide an aio_sendfile() that would deliver completion notifications via the normal aio mechanism?
>
> This race actually exists with the classical sendfile and existed
> ever since. While pages are in socket buffer, and not yet sent
> out wire, any modification to them would affect data sent.
>
> To know when you can modify data you need to use SF_SYNC flag.
> The flag should work with new sendfile as well, effectively
> making it blocking on socket. I haven't tested this, but
> should work.
>
> Adrian have also committed the kqueue notification of SF_SYNC
> event. This should work in combination with new sendfile, but
> that hasn't been tested yet. This is going to be tested soon,
> since we need this functionality.


Yup, that's exactly why I committed the sendfile kqueue completion
notification stuff.

It's main caveat is that it requires that the ethernet driver
immediately free the mbuf upon TX completion - which the chelsio
driver doesn't yet do. That has to be fixed for the notification
completion stuff to behave well under lower traffic loads.


-a