Re: "tinderbox" using stacked unionfs

From: David Naylor <naylor.b.david_at_gmail.com>
Date: Wed, 27 Jan 2010 22:58:31 +0200
On Tuesday 26 January 2010 18:41:20 David Naylor wrote:
> On Tuesday 26 January 2010 15:04:59 Gary Jennejohn wrote:
> > On Mon, 25 Jan 2010 20:39:11 +0200
> >
> > David Naylor <naylor.b.david_at_gmail.com> wrote:
> > > On Saturday 23 January 2010 12:33:14 David Naylor wrote:
> > > > Hi,
> > > >
> > > > I have been experimenting with using unionfs to build ports in a
> > > >  "tinderbox" environment.  This avoids having to remove and extract
> > > > files for the build of each port and it allows the discovery of
> > > >  installed/modified files by the port.
> > > >
> > > > Please see attached for a (updated) shell script that handles all the
> > > >  "heavy lifting" of building ports.  Of importance is LOCALBASE and
> > > >  BUILDDIR.  If you want to override LOCALBASE please use `env` as the
> > > >  script needs to know about it.  BUILDDIR (/usr/build by default) is
> > > > where the script stores everything (including PKG_DBDIR).
> > >
> > > Please see attached for an updated script.  This no longer uses `sort
> > > -u` but removed duplicates while maintaining dependency order.  (See
> > > below)
> >
> > ENOSCRIPT
> 
> See attached.  Processes are still freezing periodically (requiring a
> restart).  The mtree problem appears when installing meta ports (x11/xorg-
> apps, x11/xorg, etc).

The script finally managed to finish without stalling.   Here are some 
comparative results:

# time make -C /usr/ports/x11/xorg install clean
2185.180u 1066.927s 38:12.27 141.8%     4123+1745k 5185+1890io 51638pf+0w

# time ./ports-union-builder.sh
2264.914u 6097.779s 2:15:07.39 103.1%   3777+1608k 171735+226215io 51424pf+0w

This results in the unionfs method being ~2.5x longer in execution.  This 
difference should decrease a fair amount if the long execution times of mtree 
can be reduced.  

The benefits of using memory backed UFS storage has not been explored.  

NOTE: ports-union-builder.sh also produces packages.  

Received on Wed Jan 27 2010 - 19:58:32 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:00 UTC