Re: VM images for FreeBSD

2011/10/17 Warren Block <wblock_at_wonkity.com>

> On Mon, 17 Oct 2011, Alexander Yerenkow wrote:
>
>  Hello all.
>> I'm currently made set of scripts, which builds FreeBSD from svn sources,
>> and packing it in VirtualBox (*.vdi) compatible images.
>> It's working now, and producing something like
>>
>> FreeBSD-9-i386-r226409-2011-10-16.vdi.xz (also .vdi, .vdi.zip and plain
>> .img
>> which can be dd to USB flash).
>>
>> I'm developing this here:
>> https://github.com/yerenkow/freebsd-vm-image
>>
>> I have more goals to do (like producing more images, with a installed sets
>> of packages, like KDE-from-ports, KDE-from-area51, with experimental GEM
>> drivers etc.)
>>
>
> Excellent!  If live CD/memdisk features are added, this could also be
> useful for testing large xorg port updates before commit.
>
> PS: why bash for cron-auto-action.sh?
>

Hello all!

I'm working currently on creating images with a set pre-installed packages.
I looked at project pkgng (candidate for replacing current pkg_* subsystem),
and also I have some thought about current packages/ports system.

1. pkg_add can be launched with parameter -p $PREFIX. So, my first thought
was: I create empty directory structure with mtree, and I'll install there
all required packages; after that I need only update this installation tree
(manually by pkg_delete $old pkg_add $new, or with some tool). But I cannot
specify to pkg_add relative root, instead of real one.

Let me show example:
PKG_DBDIR=/zpool0/testroot/var/db/pkg pkg_add -p /zpool0/testroot/usr/local
ubench-0.32.tbz
installs package, and in /zpool0/testroot/var/db/pkg/ubench-0.32/+CONTENTS
there will be such record:
_at_cwd /zpool0/testroot/usr/local

I can't specify to pkg_add that it should treat /zpool0/testroot as root, as
I need (so record really should be _at_cwd /usr/local)
Instead, pkg_add allows me to make chroot, which as you understand is not
good (In specified chroot all required by pkg* binaries/libraries must
exists, unfortunately I can't specify some empty dir and install there).

Why is that? Because there is +INSTALL script in packages, in which
package/port system allows execute any code/script written by porter.

2. In ports enhancements task list (somewhere i read it) there was one item:
Make packages non-executable (or something similar). To do this properly, we
must get rid of of free-form post-install post-deinstall scripts.
To do this, we need some deep analysis of what types of actions there
happening, formalize them and provide some way to porters specify all needed
actions in Makefile.
I downloaded all packages for 9-current i386, found all +INSTALL scripts,
and kinda categorized them, you can get all of them here:
http://www.box.net/shared/ieovjj7l8omkrm3l21xb

To summarize my efforts:
I checked 21195 packages;
I found 880 install scripts;

3 scripts contains plain "exit 0"
8 install scripts contains some perl code;
17 scripts contains some additional "install" commands;
70 scripts contains some chgroup/chown actions (which probably could be done
by specifying mtree file?...)
75 contains uncategorized actions (print of license, some interactive
questions, ghostscript actions, tex, fonts etc.)
161 scripts contains some file commands, like (ld / cp / mv, creating
backups, creating configs if they aren't exists etc. )
166 scripts contains useradd/groupadd commands (many similar constructions,
not too hard to move this to .mk, in pkgng group/users can be specified in
yaml config)
380 contains pear component registration (md5 -q * | uniq  - produces
exactly one result, so these all scripts are really one, could be moved to
some pear.mk)

Why I'm interested in non-executable install of package (e.g. simple unpack
+ execute some typical actions based on package description):
- Unpacking of hundreds Mb packages takes several minutes (to mdconfig-ed
filesystem)
- Installation of these packages via pkg_add (they downloads from local ftp)
took hours in my case (to mdconfig-ed filesystem)

As you understand, to make efficient image building system, I need to deal
with package installation without spending too many cpu/disk resources.
Ideally I consider all required packages are extracted to some their own
directory, like for ubench:
$X/packages/ubench/ (and here goes all directory structure which should be
copied to new root)

plus separated info of new users/groups (maybe there need some additional
data to make package installed in such way fully working).

So, maybe someone working in this direction, or have any comments?

3. Other "ports" ideas/thoughts.
I proposed small enahcement to pkgng, but instead in pkgng this should be
implemented in ports subsystem, it's about specifying abstract dependencies,
and correct resolving of them:
https://github.com/pkgng/pkgng/issues/100

Who can comment/elaborate about this? It shouldn't be very complicated,
since currently almost same functionality provided in .mk. files ( like
USE_PERL etc)


4. Where's the "right" place to discuss ports system? :)

Thanks.

-- 
Regards,
Alexander Yerenkow