Re: Official request: Please make GNU grep the default

On Wednesday, August 18, 2010 5:54:41 pm Dimitry Andric wrote:
> On 2010-08-18 23:12, Dimitry Andric wrote:
> >> And one trial is not statistically valid - especially given the small
> >> differences.  How about multiple multiple trials with ministat.
> > 
> > The result were averages of three trials
> 
> Actually, since I kept using Doug's original grep-time-trial.sh, each of
> the three 'trials' consisted of running grep 100 times, and the listed
> time was the total elapsed time for those 100 runs.  So I assume that
> will reasonably average out the differences between each individual run?

You need the distribution, not just the averages so you can detect outliers 
and determine the standard deviation and confidence intervals.  You could use 
ministat on a file that contained all 100 runtimes perhaps.  I would use at 
least 10 trials though, 3 is a bit small.

-- 
John Baldwin