Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient

On Tuesday, September 16, 2014 12:42:36 PM Mateusz Guzik wrote:
> On Fri, Sep 12, 2014 at 09:29:56PM -0700, Peter Wemm wrote:
> > On Thursday, September 11, 2014 12:38:02 PM Patrick Kelsey wrote:
> > > On Wed, Sep 10, 2014 at 3:00 AM, Andrey Chernov <ache_at_freebsd.org> 
wrote:
> > > > On 09.09.2014 21:53, Patrick Kelsey wrote:
> > > > > I don't think it is worth the trouble, as given the larger pattern
> > > > > of
> > > > > libc routines requiring multiple capsicum rights, it seems one will
> > > > > in
> > > > > general have to have libc implementation knowledge when using it in
> > > > > concert with capsicum.  For example, consider the limitfd() routine
> > > > > in
> > > > > kdump.c, which provides rights for the TIOCGETA ioctl to be used on
> > > > > stdout so the eventual call to isatty() via printf() will work as
> > > > 
> > > > intended.
> > > > 
> > > > > I think the above kdump example is a good one for the subtle issues
> > > > > that
> > > > > can arise when using capsicum with libc.  That call to isatty() is
> > > > > via a
> > > > > widely-used internal libc routine __smakebuf().  __smakebuf() also
> > > > > calls
> > > > > __swhatbuf(), which in turn calls _fstat(), all to make sure that
> > > > > output
> > > > > to a tty is line buffered by default.  It would appear that programs
> > > > > that restrict rights on stdout without allowing CAP_IOCTL and
> > > > > CAP_FSTAT
> > > > > could be disabling the normally default line buffering when stdout
> > > > > is a
> > > > > tty.  kdump goes the distance, but dhclient does not (restricting
> > > > > stdout
> > > > > to CAP_WRITE only).
> > > > > 
> > > > > In any event, the patch attached to my first message is seeming like
> > > > > the
> > > > > way to go.
> > > > 
> > > > Well, then commit it (if capsicum team agrees).
> > > 
> > > Will do - thanks for the feedback.
> > > 
> > > -Patrick
> > 
> > Is there any possibility that this is related to the problem we've
> > recently
> > hit in the freebsd.org cluster with this month's refresh?
> > 
> > After running for a while:
> > Sep 10 02:39:44 ns2 unbound: [65258:0] notice: init module 0: validator
> > Sep 10 02:39:44 ns2 unbound: [65258:0] notice: init module 1: iterator
> > Sep 10 11:44:29 ns2 unbound: [65258:3] fatal error: event_dispatch
> > returned
> > error -1, errno is Capabilities insufficient
> > 
> > Sep 10 16:21:16 ns2 unbound: [28212:0] warning: did not exit gracefully
> > last time (65258)
> > Sep 10 16:21:16 ns2 unbound: [28213:0] notice: init module 0: validator
> > Sep 10 16:21:16 ns2 unbound: [28213:0] notice: init module 1: iterator
> > Sep 11 10:23:49 ns2 unbound: [28213:5] fatal error: event_dispatch
> > returned
> > error -1, errno is Capabilities insufficient
> > 
> > Sep 11 13:48:46 ns2 unbound: [79419:0] warning: did not exit gracefully
> > last time (28213)
> > Sep 11 13:48:46 ns2 unbound: [79420:0] notice: init module 0: validator
> > Sep 11 13:48:46 ns2 unbound: [79420:0] notice: init module 1: iterator
> > Sep 11 18:42:56 ns2 unbound: [79420:6] fatal error: event_dispatch
> > returned
> > error -1, errno is Capabilities insufficient
> > 
> > I believe this jail was started from the boot process. If I restart the
> > jail by hand from a ssh session the problem goes away.
> > 
> > This is unbound from ports and I don't have any more details than this. 
> > This is new this month.
> 
> Is this thingy multithreaded?
> 
> Currently there is a race in the kernel where fd is visible before
> relevant capabilities are installed. This can result in an error like
> this one for weird processes.
> 
> I got a patch for this:
> http://lists.freebsd.org/pipermail/freebsd-arch/2014-August/015788.html
> 
> but it got stalled on 'memory barrier' discussion. I'll try to ping
> people to move it forward.
> 
> IIRC there was a report of unbound failing this way, apparently fixed
> with aforementioned patch.

Yes, unbound is multi-threaded and your comment is the first potential 
explanation that makes sense so far.

-- 
Peter Wemm - peter_at_wemm.org; peter_at_FreeBSD.org; peter_at_yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246