Re: [HEADSUP & CFT] pkg 1.0rc1 and schedule

On Thu, Jul 12, 2012 at 11:48:41AM -0700, Doug Barton wrote:
> I do not mean this e-mail to be in any way critical. I was told after
> the new OPTIONS framework discussion that I should have asked questions
> before the change, so I'm asking these questions now; in a genuine
> attempt to get information.
> 
> On 07/12/2012 03:01 AM, Baptiste Daroussin wrote:
> 
> In the time that you have been working on this project I have asked
> numerous times for you(pl.) to answer the following questions:
> 
> 1. What are the goals for pkg?

The why part of this mail should reply this question, no?

Anyway the goal is to have a decent package manager, providing modern features:
repositories, decent dependency tracking, decent reverse dependency tracking,
managing upgrade correctly (I'll explain this more later), provide a decent
library for third party tools (desktop integration via PackageKit for example)

Providing easy package management for enterprise (who never got problems
managing packages on a large set of freebsd servers, and how complicated it is
on FreeBSD to have automated reliable puppet,salt,chef,cfengine like tools)
One of the proof of this problem is how fast people integrated pkgng in those
tools.

> 2. Why can't the existing tools fulfill those goals?

The existing tools can't fulfill those goals, because they are hardly
maintainable, the code hasn't change much since when they were written, lot of
people have tried over the year to improve them, but all of them gave up. The
design of the tools, (I mean the code) is really imho not adapted to be
improved, I spent a lot of time trying to work on it before starting a complete
new project.

For example they do not know what is a version, they do not know what are the
reverse dependencies except through this ugly hack that is +REQUIRED_BY, the
database is pretty fragile: who never got the package corrupted: empty _at_pkgdep
line for example.

> 3. How does pkg fulfill them?
> 
> You've put some of this in the various places where pkg is documented,
> but I don't see any thorough treatment of these questions. You have some
> of it below, which I'd like to see expanded on if you would be so kind. :)

It is true that, I'm not very good at documenting in general, and even more in
english, hopefully, the documentation is improving a lot recently, there is the
for usage:
http://wiki.freebsd.org/PkgPrimer
and for all other things:
http://wiki.freebsd.org/pkgng

Lot of native english speakers have joined the project and help with
documentation, if you find someting missing, do not hesitate to had the section
in the apropriate wiki page, I often have a look at them, and try to fill all
the blank section to answer user questions.
> 

> > Why pkg?
> > pkg_* tools have become hardly maintainable over the time,
> 
> I agree on this point, but the right solution (as some of us have been
> saying for years) is to move the pkg_* tools into the ports tree. You
> are correctly handling that by keeping pkg in the ports tree, I'm simply
> pointing out that this isn't a reason we need to switch to pkg.
> 
> > it lacks lots of features most of people are expecting from a package manager:
> >   - binary upgrade
> 
> I'm not sure what you mean by this. We have the ability to create binary
> packages now.

No we haven't :), I know we can mimic a binary upgrade using for example
portmaster (I describe this in a poudriere howto) but this is not fully binary
upgrade, it is deinstalling/reinstalling a package. Binary upgrade is much more
complexe than that, for example one thing you can't handle now is a package that
has been splitted into lib vs runtime will break with the current way we can do
it. Just as an example.

Just have a look at this old video:
http://www.youtube.com/watch?v=iBgcuKF8R_A (it is only 1m30)

> 
> >   - ability to search information about remote packages
> 
> This is a good feature, certainly. However there is no reason we can't
> create a tool to do this, or add the functionality to an existing tool.

Have a look at what pkgng can present you as information and you will see the
difference.

> 
> >   - real reverse dependency tracking
> >   - tracking leaves
> 
> Can you expand on what these 2 mean?

Of course. The current reverse dependency tracking in pkg_install is a hack: a
+REQUIRED_BY file trying to maintain the list of packages that may depend on the
said dependency, a good way to see it is a hack is to see how often the file get
broken (and on portmaster you added an options to fix them so you might know :))

> 
> What I'm looking for is compelling motivation to make this overwhelming
> change to the ports infrastructure.

There is not much changes needed in the ports infrastructure. It now works ootb
But there are tons of improvements pkgng will offers, like: ability to simply
add new plist keyword without the need of modifying pkgng, native support for
registering a package from a stagedir.
> 
> 
> > Schedule
> > --------
> > 
> > The plan is to switch the ports tree to pkgng on CURRENT by default on July 25th
> > No dates are planned yet for other branches.
> 
> Can you describe how this is going to be done? I assume with an
> OSVERSION knob in bsd.port.mk?

Yes the plan if to check OSVERSION and define WITH_PKGNG on the concerned
versions except if the user have defined NO_PKGNG in make.conf

> 
> > Note that there will be a NO_PKGNG knob for some time (undefined yet) for people
> > not will to switch on July 25th
> > 
> > Please also note that some ports won't work with pkgng right now, because pkgng
> > is more strict than pkg_install on purpose.
> > The major one is: nvidia drivers, because pkgng does not allow to overwrite a file
> > owned by another package, and we will not accept any hacks for that in pkgng.
> 
> IMO it would be a very large mistake to switch the default in any branch
> until the problem with the nvidia drivers is sorted out. We have a lot
> of users (myself included) who use this port, and by switching the
> default there's going to be 1 of 2 outcomes for those users. Either they
> will opt-out, which means you won't get the level of testing you're
> looking for; or you'll break their existing ports installation. Neither
> outcome is desirable.

I proposed 3 differents way to fix the nvidia driver problem, no one really
takle the task on it even if most agreed with at least one of the method.

So waiting for someone to provide a really clean way to provide nvidia driver,
I'm now working on a small modification of the current way it works, that will
bypass the pkgng strictness.

So nvidia-driver should be fixed on pkgng quite soon (if maintainer do accept
the modification :)).

I hope, I have answered your question correctly?

regards,
Bapt