Re: "tinderbox" using stacked unionfs

On Saturday 23 January 2010 12:33:14 David Naylor wrote:
> Hi,
> 
> I have been experimenting with using unionfs to build ports in a
>  "tinderbox" environment.  This avoids having to remove and extract files
>  for the build of each port and it allows the discovery of
>  installed/modified files by the port.
> 
> Please see attached for a (updated) shell script that handles all the
>  "heavy lifting" of building ports.  Of importance is LOCALBASE and
>  BUILDDIR.  If you want to override LOCALBASE please use `env` as the
>  script needs to know about it.  BUILDDIR (/usr/build by default) is where
>  the script stores everything (including PKG_DBDIR).

Please see attached for an updated script.  This no longer uses `sort -u` but 
removed duplicates while maintaining dependency order.  (See below)

> # env LOCALBASE=/tmp/local BUILDDIR=/tmp/build ./ports-union-builder.sh
> 
> Will install x11/xorg without affecting already installed systems.
> 
> CURRENT STATUS:
>  - *** Currently kernel stack size is too small and the above will trigger
>  a stack overflow.  Recompiling a kernel with ``options KSTACK_PAGES=32''
>  will alleviate that problem.

In building xorg there were at least 207 stacked unionfs (206 ro, 1 rw, all 
noatime).  

>  - Currently there is a build problem that affects eggdbus/polkit (possibly
> others) thus preventing x11/xorg from being built.  I will investigate the
> cause (help welcome).

This is due to not mounting the dependencies in the correct order.  If 
dependency 'a' modified file from dependency 'b' then mounting in order 'a', 
'b' will result in those changes being lost as the original files from 'b' 
will supersede 'a'.  The dependencies need to be mounted 'b', 'a'.  

This has been fixed although may cause a problem if multiple "independent" 
ports modify the same file.  This is a detectable problem.  

>  - I highly recommend running this in a chroot
>  - NO WARRANTY, SLIPPERY WHEN WET, EATS CHILDREN.

 - I am experiencing process freeze (anything involved in the directories that 
are unionfs).  Any way that I can figure out the problem?  I can run a kernel 
will full debug capability.  
 - mtree does not behave well with unionfs and consumed a fair amount of 
resources:
# /usr/sbin/mtree -U -f /usr/ports/Templates/BSD.local.dist -d -e -p 
/usr/local/
took a long time (many minutes) to complete.  

> Since the script doesn't complete a full build I am unable to compare the
> build speeds (thus the overhead of unionfs) but here are some partial
>  results (with FORCE_MAKE_JOBS and quad core):

The script is now able to complete building but not at once due to process 
freezing.

Please help with debugging the process freezing.  (There is a LOR I have 
reported for unionfs: kern/141950)

Regards

David