Re: Interesting speed benchmarks

On Fri, 2007-Jan-26 18:33:39 +0100, Oliver Fromme wrote:
>Robert Huff wrote:
> >         $DUMP_LEVEL -D $DUMPDATES_FILE -Lau -f 
> > 
> >         gets me 2 +/- 0.2 mbytes/sec.
> >         Is this a reasonable value?  (I.e. is dump the limiting
> > factor?)  Do I need to reconfigure something, or is my hardware just
> > lame?

I presume you are dumping an internal SCSI disk onto a USB disk.  Dump
is slow but shouldn't be that slow.  Doing a dump of root, I get
10MB/s on one system and 15MB/s on another.  Even my P-120 firewall
gets 2.7MB/s and it is actually CPU limited, not disk limited.  What
do you get if you do the dump to /dev/null?

>Historically the performance of dump(8) has always been
>quite bad.  It didn't matter in ancient times because
>tape drives were slow back then, so dump(8) was not the
>limiting factor.  ;-)

Tapes ceased to be the limiting factor long ago.  There was a thread
bemoaning the "Glacial speed of dump backups" on FreeBSD 4.6 in mid-
2002.  I thought there was an earlier thread but I don't have my old
mail archives to hand.  There are two factors that make dump's
performance abyssmal:
1) It physically re-reads the block containing the inode being dumped
   multiple times (to ensure that the file hasn't changed size)
2) It never reads more than the FS blocksize (8K or 16K).

The first is not relevant for dumping snapshots and could probably be
disabled.  Caching (-C option) was introduced to attempt to alleviate
the second but may or may not improve the overall throughput (it
depends on the actual file layouts).  Probably the overall design of
dump needs a major re-think to try and improve performance - the
introduction of snapshots removes the need to handle dynamic file-
systems.

>For that reason I try to avoid dump(8) when possible.
>Maybe you should try to use tar, cpio or pax instead and
>check if the performance is better.

Note that dump/restore is the only tool that can correctly reproduce
sparse files.  tar, cpio and pax also have filename and file size
restrictions.  I don't think that cpio or pax support ACLs or file
flags.

-- 
Peter Jeremy