Re: [HEADSUP] No more pkg_install on HEAD by default

From: Teske, Devin <Devin.Teske_at_fisglobal.com>
Date: Sat, 13 Jul 2013 18:54:21 +0000
On Jul 13, 2013, at 11:16 AM, Kimmo Paasiala wrote:

> On Sat, Jul 13, 2013 at 7:46 PM, Teske, Devin <Devin.Teske_at_fisglobal.com> wrote:
>> 
>> On Jul 13, 2013, at 8:17 AM, Teske, Devin wrote:
>> 
>> 
>> On Jul 13, 2013, at 1:07 AM, Baptiste Daroussin wrote:
>> 
>> On Fri, Jul 12, 2013 at 11:52:19PM +0000, Teske, Devin wrote:
>> On Jul 12, 2013, at 4:16 PM, Baptiste Daroussin wrote:
>> 
>> Hi,
>> 
>> I have just committed (r253305) a change the make pkg_install not being built
>> and installed by default on HEAD.
>> 
>> If you are still relying on it, be careful and add WITH_PKGTOOLS=yes in your
>> src.conf(5)
>> 
>> [snip]
>> 
>> I for one am effected and will have to change things.
>> 
>> If you are referring to bsdconfig's package management,
>> 
>> [snip] Yes. that's what I'm talking about. [snip]
>> 
>> 
>> it is not working anyway
>> HEAD as we do not and will not provides any pkg_install for HEAD via any of the
>> usual distribution process: http, ftp, CD.
>> 
>> 
>> 
>> The FTP mirror of packages is going away? (if you said yes and pointed at a prior conversation about leading up to this, I would not be surprised -- I'm just questioning it because I don't see the value in pairing-down methods of acquisition)
>> 
>> If this is the case, what's the surviving acquisition method? A custom TCP protocol perhaps?
>> 
>> There may be those that wish to use pkg in the pkg_add manner and download things and then inspect them manually before adding them. For example, I often go the freshports.org<https://urldefense.proofpoint.com/v1/url?u=http://freshports.org/&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=Ft5J2W3Nm8xze74zARHsiglLFTGOYrxAzaubbP7wvcM%3D%0A&s=cb1df85e237d1a6be19f981e8b352419dd056ea0f4e782b330cb9c7cfd15fda5> to find a package that fills a need... download it from the official FTP server(s), inspect all of them, and choose the one that best fits me need (and only then installing from the local file; tossing the rest). If I go through the "pkg" tool, I have to inspect things *after* they've been installed which is not to my satisfaction.
>> 
>> 
>> [snip
>> bsdconfig is not installed by
>> default,
>> 
>> Wrong, please see...
>> http://svnweb.freebsd.org/base?view=revision&revision=252862
>> [snip]
>> 
>> The first thing that comes to mind in reprogramming bsdconfig's package management for pkgng...
>> 
>> We have a *very* large list of FTP mirrors. This will presumably be replaced with a list of "pkg" mirrors.
>> 
>> Do we have such a list that we want to program into the base configuration of bsdconfig?
>> --
>> Devin
>> 
> 
> Come on, this only concerns 10-CURRENT.

Right; which is why -current is CC'd.


> Where is it stated that the
> FTP mirrors for FreeBSD 8/9 packages are going away?
> 

I don't know where you got the impression that I was concerned with 8/9 packages.

I'm strictly concerned with HEAD and strictly concerned with planning for the future.

So yes... I'm asking... in a HEAD world, what is the "officially supported" method of acquisition?

I saw mention on a page that "rsync access will provided for those who want to mirror" but my I'm not really interested in mirroring while I would like to continue to be able to grab packages myself without installing them.

Maybe the answer is that "pkg" has a method of acquiring a single package (without dependencies) without downloading it. That would solve my personal workflow issue (again, I like to download tarballs, inspect them, and then install them from the locally downloaded file -- then if it passes validation tests, that downloaded file is migrated to our own distribution servers; it's a work-flow for validating packages -- it has less to do with how pkgng works and more just whether I can get packages in a fashion such as FTP or HTTP or whatever.

Now that aside, bsdconfig currently is a different story. To rewrite the packages module to work in a HEAD world for HEAD, using pkgng servers, I'm going to have to change the way things are done. Currently there is an abstraction layer for fetching packages from media (which can be FTP, HTTP, HTTP Proxy, NFS, Local Directory, CDROM, a DOS partition, a UFS partition, and [god forbid] Floppy -- with perhaps SMB/CIFS down the road). When it wants to add a package, it calls the routine to get the package data by name on standard-out and pumps it into pkg_add (after of course initializing the media).

This abstraction layer is useful to pkgng as it prepares the source. What I *suspect* will have to stop happening is the direct-fetching of the package data and piping that into a program (however, I *could* continue to do that .. "pkg" supports adding of local packages and iirc supports reading from standard-in). Instead, perhaps just call "pkg" in a small-handful of different ways (pointed at an ftp server; pointed at a local file; pointed at http; etc.). But the preparation of an NFS mount would be handled by the abstraction layer leading up to the calling of pkg.

That issue aside (whether to have pkg do the bidding or to use the existing abstraction framework to fetch the file), there are benefits to each route.

I'm leaning the latter route (of having bsdconfig's abstraction layer do the fetching and simply have pkg add a file from stdin) because I can then write a program that is essentially a more advanced version of sysutils/pv (and would be BSD licensed).

There are certainly things that pkg can excel at that the latter method may by-pass (which might make it undesirable). For example, bsdconfig currently reads the INDEX file and handles dependencies itself. It could continue to do this, but we want pkg to handle the dependencies.

If I chose to continue to do things in this manner, it would be bad (I estimate) because I believe that shoving dependencies into pkg in an ordered fashion before adding the requested package would ultimately cause the dependencies to _not_ be recognized as leaves (correct me if I'm wrong), meaning that something like "cutleaves" wouldn't be able to prune them from the three after the dependencies have been gone.

So to take full advantage of all the features of pkg, I think I'll relegate the idea of a sysutils/pv to working for the distfetch side of things (another topic) and just wield pkg in one of a small-handful of ways pointed directly at media.

Which brings us back around to the question at hand...

Does it support FTP? HTTP? HTTP Proxy? Local File? (that's about all that I need -- the last-one... Local File would be for repositories mounted via the preparation-framework which is part of the abstraction layer -- we'd just ignore the acquisition methods thereof).

If FTP access (or any of the other remote access methods) are going away for HEAD pkg access, I'll need to know so I can make the appropriate changes in the HEAD branch of bsdconfig.
-- 
Devin

_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
Received on Sat Jul 13 2013 - 16:54:24 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:39 UTC