Re: Is fork() hook ever possible?

On Sat, Nov 12, 2011, Andrey Chernov wrote:
> On Sat, Nov 12, 2011 at 10:41:35AM -0500, David Schultz wrote:
> > On Sat, Nov 12, 2011, Andrey Chernov wrote:
> > > On Tue, Sep 16, 2008 at 04:19:32PM -0400, David Schultz wrote:
> > > > secteam_at_ already agreed to the idea of solving the fork problem as
> > > > in OpenBSD over a month ago. 
> > > 
> > > On Wed, Sep 17, 2008 at 12:50:25PM +0400, Andrey Chernov wrote:
> > > > I agree with your patch (BTW you can remove unneded #define RANDOMDEV).
> > > 
> > > The question remains: why you don't commit this patch all that 3 
> > > years, having secteam_at_ and mine agreements too?
> > 
> > Sorry, but in the three years that have intervened, my brain has
> > paged out the relevant context.  As I recall, there were issues
> > with some of your changes to arc4random() and I proposed tracking
> > OpenBSD's implementation more closely.  
> 
> I can't say for secteam_at_ side (it was you who said that they agree), but 
> personally me still agree with your proposal and still see security 
> problem in our current implementation, like the same-generated tmp names 
> after fork in son and parent.
> 
> > If everyone's in agreement on that, please go ahead and commit the changes.
> 
> I can't... It seems I reach dead end talking to our _at_secteam. In few 
> words, they:
> 1) Explicitly disallow my commits in all 'random' areas until their 
> review.
> 2) They never do that review (I must to mention again that 3 years 
> passed since they promise it).
> Being particular, I suggest them to use your patch at the end. Nothing 
> happens.
> Hope you'll get more luck with them committing it by yourself.

I don't have those patches anymore, but I redid them from scratch
using the latest revision from OpenBSD.  The patch at
http://www.freebsd.org/~das/patches/vshead.diff syncs our
arc4random.c with OpenBSD's to the extent possible, style bugs and
all.  It seems prudent to treat it as a vendor source and avoid
gratuitous differences:  Unlike our version, all the changes to
OpenBSD's arc4random.c were vetted by several people.  Switching
to OpenBSD's version fixes the bug where a parent and child
process see the same random sequence.

The diff http://www.freebsd.org/~das/patches/vsobsd.diff shows the
differences between our version and OpenBSD's after applying the
patch.  Most of the remaining differences are in arc4_stir(),
where we use /dev/random instead of a sysctl to get entropy for
the IV.  In arc4_stir(), I also fixed the bug where the wrong
buffer size was being passed to arc4_addrandom(), resulting in
entropy loss.  That change should be committed separately.

When adding the XXX comment in the patch, I noticed that it isn't
clearly documented what arc4random() is supposed to provide, and
there seems to be a substantial amount of confusion on this point.
I can think of three reasonable kinds of RNGs:

  1. Low-quality pseudorandomness, e.g., for test generation.
     These are only "random" w.r.t. certain statistical tests.

  2. High-quality, reproducible pseudorandomness, e.g., for
     deterministic encryption, or test generation with stronger
     statistical requirements.

  3. High-quality, unpredictable pseudorandomness, e.g., for key
     generation.

David Mazieres' original arc4random() implementation was intended
to provide a less useful variant of #2, but was extended to
provide #3 in OpenBSD.  FreeBSD's version provides #3 most of the
time, unless something goes wrong -- e.g., you forget to load
random.ko or you run it in a jail where there's no /dev -- and
then you get #2.  It seems that most people think arc4random() is
giving them #3, and if that's the case, then arc4random() should
be fail-fast: it should abort if it's unable to obtain a
high-quality IV.

There's similar confusion evident in the revision history for
random(), which really only provides #1.

> On Tue, Sep 16, 2008 at 04:19:32PM -0400, David Schultz wrote:
> > If getpid() really winds up being a serious problem, we can solve
[...]
> I run some tests but can't come to conclusion, is overhead is significant 
> or not for real life tasks (which usually don't call arc4random() very 
> often in the loop).

I have a bunch of tests for floating point that use arc4random(),
and they are substantially slower with the getpid() changes.
Realistically they should be using mrand48(), though.

Here's a TODO list of worthwhile improvements that I don't have
time for.  None of them seem terribly pressing.

 - Document what arc4random() actually provides.  (The OpenBSD
   manpage is pretty good, albeit a bit technical.)

 - Like OpenBSD, implement and use a sysctl to get the IV in
   arc4random() instead of /dev/random.  This is likely to be
   faster, and there's less than can go wrong (e.g., jails without
   a working /dev).

 - Make arc4random() abort if it can't get any cryptographic-quality
   entropy.  Make sure it's still possible to boot in single-user
   mode without random.ko.

 - Either optimize getpid() or replace its use with a fork hook,
   so that it isn't necessary to do a syscall on every
   arc4random() call.  If there's no speed advantage to using
   arc4random(), people might as well just be reading from
   /dev/random all the time.