Re: CFT: FreeBSD Package Base

On Mon, 29 Apr 2019 09:25:05 -0400
Kris Moore <kris_at_ixsystems.com> wrote:

> On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot <manu_at_bidouilliste.com>
> wrote:
> 
> >
> >  Hi Kris,
> >
> > On Sun, 28 Apr 2019 15:52:21 -0400
> > <kris_at_ixsystems.com> wrote:
> >
> > > FreeBSD Community,
> > >
> > >
> > >
> > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and
> > 13-current
> > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images
> > which
> > > will allow users to perform all updating via the 'pkg' command directly.
> > > Rather than trying to answer all questions in this announcement, we've
> > > created a FAQ page with more details. Please refer to this page, and let
> > us
> > > know if you have additional questions that we can include on that page
> > going
> > > forward.
> > >
> >
> >  While I appreciate the effort I have some doubt about your
> > "re-implementation" of pkgbase. I don't see any improvement compared to
> > what is in base currently, I even see downside of your implementation.
> >
> >  - How do you plan with the need of updating kernel first, reboot and
> > updating the rest of the userland after ? (Needed for major and minor
> > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and
> > -HEAD branch). This is still a problem with the base pkgbase.
> >
> 
> We've written our own tool "sysutils/sysup" in GO which handles this. It
> performs updates using Boot-Environments to ensure that kernel/world are
> updated at same time.
> 

 Which could never be imported into FreeBSD.

> 
> 
> >  - This is even worse because you are using the same repository for
> > base and pkg so if a user pkg update and both kernel and pkg(8) needs
> > to be updated and pkg use a new syscall or capsicum thing it will be
> > updated first and couldn't proceed with the rest of the update (this is
> > a supposition, I haven't personally tested).
> >
> 
> See above.
> 
> 
> >  - It seems that multiple kernels isn't supported in your
> > implementation, this is already supported in pkgbase but still need
> > some love. This is an important point as it will allow user to choose
> > easily the kernel that they want to use and will also allow us
> > developper to push kernels with new features to help testing.
> >
> 
> Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get
> the Witness-enabled kernel installed alongside non-debugging one.

 Mhm no, the kernel-debug packages only add the debug file
in /usr/lib/debug/boot/
 I'm talking about installing multiple kernels in //
(i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like
describe here :
https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues
in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point.

> 
> >  - Since you reduced the granularity on the userland bits it would mean
> > that if we use your implementation for -p updates we would download the
> > whole userland packages instead of just updating the package that was
> > patched. For example with pkgbase, updating from 12.0 to 12.0p1 will
> > only update the FreeBSD-runtime package. Yes this package is still big
> > to download when you compare to what have changed but until pkg(8) have
> > delta pkg supports (and if it will have support, I don't know if
> > this is a wish or not) this is the best way to go.
> >
> 
> Correct, this is by design. We used the in-tree pkg base for nearly a year,
> and found that the granularity didn't really offer any savings from a
> download or time perspective. Updating 100+ packages took far longer than a
> single one, due to all the meta operations. Additionally in real-world
> usage, we found that base packages tended to all get updated at the same
> time, which took far longer via pkg, since it had to go and perform 100+
> fetch operations just to download the base system bits.
> 

 But you never need to update 100+ packages on a proper pkgbase setup
for -p updates.
 Again on a 12.0 to 12.0-p1 update only one package will be updated.

> 
> >  - I see that you are sorting the plist for kernel and userland based
> > on the line length [1], why is that ?
> 
> 
> Whoops! I'll fix :)
> 
> 
> >
> >  I think that the only advantage that your solution offers is that if
> > we remove a componant of base (rcmds for example in 12-CURRENT) those
> > files would be removed as they are in the userland-base package while
> > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and
> > will not be deleted in the user computer.
> >
> 
> 
> Correct, this is one of the things which prompted us to go this direction.
> Being able to handle crazy mixed WITH/WITHOUT flags was important to us,
> current pkg base did not handle that so gracefully. 

 Can you give me more info on this ? What where the WITH/WITHOUT flags
that causes problems ?

> Additionally we've
> added some additional features, such as being able to 'pkg install os/src'
> to get system sources used in exact build, as well as being able to rebuild
> your local world / kernel packages using ports "make config" framework is
> super handy.
> 

 That would not be hard to add to pkgbase, I'll add it to my todo list.

> >
> > >
> > > Additionally, I will be hosting a Package Base working group at BSDCan
> > 2019,
> > > and welcome user and developer attendance to discuss this and other
> > ongoing
> > > package work:
> > >
> > >
> > >
> > > https://wiki.freebsd.org/DevSummit/201905/PackageBase
> > >
> >
> >  I will be present and looking forward to work with you on this.
> >
> >  Cheers,
> >
> >  P.S. :  FYI I'm working on pkgbase currently and I will have some
> > patches to commit soon (bsdinstall support, memstick creation that
> > install a pkgbase aware installaton etc ...).
> >
> 
> Great! Looking forward to discussion then!
> 
> 
> >
> > [1] :
> >
> > https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35
> >
> > --
> > Emmanuel Vadot <manu_at_bidouilliste.com> <manu_at_freebsd.org>
> >
> _______________________________________________
> freebsd-stable_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe_at_freebsd.org"


-- 
Emmanuel Vadot <manu_at_bidouilliste.com> <manu_at_freebsd.org>