Re: "tinderbox" using stacked unionfs

Hi David,

I like your unionfs idea. Noticing the tinderbox limitation I moved on my zfs 
based system to a jail I set up using a self written script. It is currently 
customized to my environment but still might prove useful for quick setup of 
jails.

it handles 
* fs creation 
* putting the proper data on them
* routing setup (since I use a laptop and my active nic changes from cable-
bound to wireless)

Since this both contributes to the topic of building software in a separate 
environment, I share it here. 

Regards

Christof

Am Montag 25 Januar 2010 19:39:11 schrieb David Naylor:
> On Saturday 23 January 2010 12:33:14 David Naylor wrote:
> > Hi,
> >
> > I have been experimenting with using unionfs to build ports in a
> >  "tinderbox" environment.  This avoids having to remove and extract files
> >  for the build of each port and it allows the discovery of
> >  installed/modified files by the port.
> >
> > Please see attached for a (updated) shell script that handles all the
> >  "heavy lifting" of building ports.  Of importance is LOCALBASE and
> >  BUILDDIR.  If you want to override LOCALBASE please use `env` as the
> >  script needs to know about it.  BUILDDIR (/usr/build by default) is
> > where the script stores everything (including PKG_DBDIR).
> 
> Please see attached for an updated script.  This no longer uses `sort -u`
>  but removed duplicates while maintaining dependency order.  (See below)
> 
> > # env LOCALBASE=/tmp/local BUILDDIR=/tmp/build ./ports-union-builder.sh
> >
> > Will install x11/xorg without affecting already installed systems.
> >
> > CURRENT STATUS:
> >  - *** Currently kernel stack size is too small and the above will
> > trigger a stack overflow.  Recompiling a kernel with ``options
> > KSTACK_PAGES=32'' will alleviate that problem.
> 
> In building xorg there were at least 207 stacked unionfs (206 ro, 1 rw, all
> noatime).
> 
> >  - Currently there is a build problem that affects eggdbus/polkit
> > (possibly others) thus preventing x11/xorg from being built.  I will
> > investigate the cause (help welcome).
> 
> This is due to not mounting the dependencies in the correct order.  If
> dependency 'a' modified file from dependency 'b' then mounting in order
>  'a', 'b' will result in those changes being lost as the original files
>  from 'b' will supersede 'a'.  The dependencies need to be mounted 'b',
>  'a'.
> 
> This has been fixed although may cause a problem if multiple "independent"
> ports modify the same file.  This is a detectable problem.
> 

> >  - I highly recommend running this in a chroot
> >  - NO WARRANTY, SLIPPERY WHEN WET, EATS CHILDREN.
> 
>  - I am experiencing process freeze (anything involved in the directories
>  that are unionfs).  Any way that I can figure out the problem?  I can run
>  a kernel will full debug capability.
>  - mtree does not behave well with unionfs and consumed a fair amount of
> resources:
> # /usr/sbin/mtree -U -f /usr/ports/Templates/BSD.local.dist -d -e -p
> /usr/local/
> took a long time (many minutes) to complete.
> 
> > Since the script doesn't complete a full build I am unable to compare the
> > build speeds (thus the overhead of unionfs) but here are some partial
> >  results (with FORCE_MAKE_JOBS and quad core):
> 
> The script is now able to complete building but not at once due to process
> freezing.
> 
> Please help with debugging the process freezing.  (There is a LOR I have
> reported for unionfs: kern/141950)
> 
> Regards
> 
> David
> 


-- 
()  ascii ribbon campaign - against html e-mail 
/\  www.asciiribbon.org   - against proprietary attachments