Re: Why we don't use bzip2 in sysinstall/rescue?

Max Laier schrieb:
> With an amd64 world(236M) tar'ed together with -czf / -cyf respectively I 
> get the following input bandwidth numbers (gathered via dd 
> if=amd64.t{g,b}z of=/dev/stdio | {g,b}zcat > /dev/null):
> 
> hw.model: Intel(R) Pentium(R) 4 CPU 2.00GHz:
> x bz   + gz
>    N           Min           Max        Median           Avg        Stddev
> x 13       1411432       1554463       1544446     1521445.7     50394.705
> + 13      13391136      13847045      13804007      13726781     138363.27

Instead of measuring data rates on the input side (compressed)
I think what matters is the output data rate, since that is what
needs to be written to disk during an installation.

If I assume a compression by a factor of 4 for bzip2, the data
rate will be some 6MB/s after decompression on a P4/2GHz and
14MB/s on the Opteron 275. I have measured an P3/733 to deliver
3.25MB/s for bzip2 (and 40MB/s for gzip with a compression of
about 1:3.5).

> Difference at 95.0% confidence
>         1.22053e+07 +/- 84296.2
>         802.22% +/- 5.54053%
>         (Student's t, pooled s = 104125)
> 
> hw.model=AMD Opteron(tm) Processor 275:
> x fast.bz  + fast.gz
>    N           Min           Max        Median           Avg        Stddev
> x 10       3429556       3889574       3449869     3525725.6     169675.46
> + 10      41967910      46046387      45944662      45490300     1257435.8
> Difference at 95.0% confidence
>         4.19646e+07 +/- 843005
>         1190.24% +/- 23.9101%
>         (Student's t, pooled s = 897200)
> 
> So it seems that bzip2 will indeed be bound to CPU - at least when 
> installing from CD.  netinst over the internet is a different story, 
> though.

Since lots of small files are written, we have to consider the
transaction rate of the disk and file system being written to.

And while 3MB/s is a little low, 6MB/s does not look unreasonable
and 14MB/s is definitely sufficient to make the target disk drive
become the limiting component (except when you install to a RAM
disk or to a RAID storage system with gigabytes of cache ;-)

Regards, STefan