[HEADSUP][CFT] pkgng beta1 is out

From: Baptiste Daroussin <bapt_at_FreeBSD.org>
Date: Mon, 30 Jan 2012 13:39:30 +0100
Hi,

pkgng has just reached the beta phase, and has now found its way to the
ports tree (disabled by default).

1/ Why pkgng?
------------

Our current pkg_install tools are showing their age, are hard to maintain,
and they lack features:

- missing metadata
- no upgrade support
- no repository support
- no fine dependency tracking
- no modern binary package management
- and many others

Having old tools makes it hard to improve the ports infrastructure, as a
result lots of hacks have found their way into the different Mk/bsd.*.mk
files to work around pkg_install limitations plus there are lots of hacks
in the packages metadata itself such as _at_comment which are not comments,
and so forth.

We have people writing tools to improve the situation (portmaster and
portupgrade to name two), but they are limited by and can become quite
complicated to maintain because of the pkg_install limitations.

2/ What it is?
--------------

It is a tool that is designed to replace pkg_install and provide modern
features to advance package management on FreeBSD.

It has been done with compatibility in mind.  Most of the ports tree are
able to build on pkgng without modification (21500 successful packages is
the highest pkgng score so far).  The missing ones will be easily fixed
with pkgng in ports.

It has been done with ease of migration in mind.  It is easy to migrate
from pkg_install to pkgng.  (Please note that going backwards is not
possible.)

It has been done with FreeBSD features in mind: it supports chroot, jails,
rcng, etc.

It has been done with scripting features in mind: 'pkg query' will allow
you to query almost everything from the pkgng database in a script friendly
way.

It has been done with improvement in mind: it doesn't require a privileged
account to create packages with root files in it; it is already able to
package from a stage/fakeroot/name_it_like_you_want directory; it is also
able to fake the package creation to directly install the package from that
fake/stage/whatever directory.

It has been done with human readability in mind: the new metadata is
stored in YAML format; the plist keywords can be extended with YAML (for
the ports).

It has been our thinking that the pkg binary is not able to please
everyone's needs, so it has been written on top of a library which can be
used by any other third party tools.  (Think about packagekit, or ruby
binding for portupgrade for example, or any other usage like these).

pkgng is the result of my long studies and reflection about packaging
(studying what is done elsewhere: apt/dpkg, yum/rpm, pacman, aix, solaris,
netbsd, openbsd) and how to have something that tries to take the good
ideas from them, but tries not to take the *over engineered* complicated
parts. And most importantly, tries to do it the FreeBSD way: which means
it should work with the ports tree as-is (and help improve it in the future).

3/ Roadmap
----------

We plan a very long beta phase with lots of beta versions, released as
often as possible to ease testing and help improve the tool as much as
possible.

The goal, now that we are in beta is to not break anything for users, which
means that pkgng will be able to safely upgrade itself.  (No real breakage
occurred during the alpha phase; expect even less in beta.)

Most of the big features are implemented, so now if you have a
revolutionary idea that breaks everything, it won't find its way into pkgng
1.0.  You can still provide it for pkgng 2.0.

1.0 is not revolutionary because of the way that it is full of workarounds
to allow compatibility with the current ports tree.  At some future time
(TBD), once we have dropped pkg_install support, things will be able to
move forward faster.

pkgng will live in the ports tree, so it will evolve with the
infrastructure, allowing us not to have to wait for the EOL of a release to
be able to move forward to new features.

The library API is currently not considered stable; it will be designated
stable as of pkgng 2.0.  Therefore, if you are going to use the library in
a third party project, you can expect some breakage from time to time.  Of
course, we will avoid breakage as much as possible.

The plan is to have pkgng 1.0 ready and rock solid for 10.0-RELEASE and
9.1-RELEASE.

The more testers/contributors we have, the faster we can go, and the faster
we go, the faster we can drop pkg_install and improve our port
infrastructure.

(Note: due to limitations in FreeBSD 7.x, we do not plan to backport
there.)

4/ pkgng itself
----------------

pkg add: add packages the old way (should be avoided by users)

pkg audit: audit the installed packages for vulnerabilities

pkg autoremove: interactively propose packages to be removed that were
installed automatically (as a dependency) and not depended on anymore

pkg check: check the installed packages database, prompting for
inconsistency and proposing to try to fix it

pkg clean: cleanup the package cache from binary installation (from
repositories)

pkg delete: deinstall packages

pkg info: query information from the installed packages in a user-friendly
way

pkg install: install packages from a remote repository

pkg query: query information from the installed packages in a
script-friendly way

pkg register: register packages in the database, synchronising the files
from a fake/stage directory, or with already installed files (like
currently with ports)

pkg search: search the remote database for packages

pkg update: update the remote repositories databases

pkg updating: scan the installed ports and show all UPDATING entries that
affect one of the installed ports (same as old pkg_updating)

pkg upgrade: perform a full binary upgrade of the installed packages

pkg version: determine whether package(s) need to be updated (same as old
pkg_version)

pkg which: determine which package owns a file

pkg2ng: convert a pkg_install installed database to a pkgng installed
database (you would need the ports tree for pkg2ng to be able to gather
missing information about packages)


Sample output of pkg info:

$ pkg info -f libreoffice:
Name           : libreoffice
Version        : 3.4.4
Origin         : editors/libreoffice
Prefix         : /usr/local
Categories     : editors
Licenses       : MPL & LGPL3
Maintainer     : office_at_FreeBSD.org
WWW            : http://www.libreoffice.org/
Comment        : Full integrated office productivity suite
Options        :
        DEBUG: off
        GNOME: off
        GTK: on
        JAVA: off
        KDE4: off
        MMEDIA: off
        PYUNO: off
        SDK: off
        SYSTRAY: off
        WEBDAV: off
Flat size      : 319 MB
Description    :
LibreOffice is the free power-packed Open Source personal productivity suite for
Windows, Macintosh and Linux, that gives you six feature-rich applications for
all your document production and data processing needs: Writer, Calc, Impress,
Draw, Math and Base.

WWW:    http://www.libreoffice.org/

5/ what is missing
------------------

- for 1.0:

 * currently the user handling is done using the _at_exec/_at_unexec scripts from
 the ports; we need to finish the switch to the pw_/gr_util API to have
 cleaner handling.

 * better error reporting; lots of error/warning messages are currently
 technical and need to be improved to become more user-friendly.

 * better documentation; pkgng does a lot of things (more than what is
 described here) and needs to be documented.  We lack enough native English
 speakers that are able to correctly document everything.

 * bug hunting and fixing.

- for future:

 * capsicum: during EuroBSDCon some ideas were shared on how we can
 capsicumize pkgng, and more generally, increase security in package
 management; we need to have more thought about this subject.

 * improved arch support; currently pkgng is able to prevent installing
 amd64 packages into an arm box for examples, but we can go further and
 have real arch handling -- packaging noarch packages only once for all
 architectures (shell scripts, data, etc) and share them between the
 repository.  This would reduce the size of the repositories and speed up
 the packages building process.

 * incrementally updated packages (diff packages): this is a really
 complicated task but could be done, would need a clean design.

 * abstract/alternative packages (e.g. "provide http_server")

 * having a real sat solver for the dependency tree.  Currently we have a
 really simple and minimalistic solver which works well but if we can to go
 to an even finer package management we would need a real solver.

 * many more to join the project to share your ideas.


Keep in mind that in pkgng, every thing should be and will remain simple, so
when you come up with ideas, try providing a simple and clean design :)

to use pkgng:

echo WITH_PKGNG=yes >> /etc/make.conf
make -C /usr/ports/ports-mgmt/pkg install clean

Some links: 
http://wiki.FreeBSD.org/pkgng
http://github.com/pkgng/pkgng

Note that on github you can find a patch for portmaster (against 3.10)

On behalf of the pkgng team
Bapt

Received on Mon Jan 30 2012 - 11:39:35 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:23 UTC