On 08/23/2012 16:31, olli hauer wrote: > On 2012-08-23 21:50, Kris Moore wrote: >> On 08/23/2012 13:10, Jeremy Messenger wrote: >>> On Thu, Aug 23, 2012 at 11:49 AM, Kris Moore <kris_at_pcbsd.org> wrote: >>>> On 08/23/2012 12:26, Jeffrey Bouquet wrote: >>>>> I am following with dread the planned implementation of the deprecation of /var/db/pkg as a package registry... I use each /var/db/pkg directory as a database into the port installation/status, using sed/grep/portmaster/portmanager/.sh scripts/find/pipes etc... to fix stuff. For instance, an upgrade py26 > py27. >>>>> cd /var/db/pkg >>>>> ls -lac | grep py26 >>>>> ls -lac | grep python >>>>> as the more simple example. >>>>> .... >>>>> With due respect to its developers and the persons who agree that >>>>> the package tools could be upgraded, the mandatory >>>>> usage of a front-end database to a file directory one >>>>> is here viewd as mutt-only-mbox, registry-and-bsod rather >>>>> than /etc/local/rc files, deprecation of sed/grep/find/locate/.sh/portmaster/portmanager as tools to fixup/upgrade the ports that are registered; >>>>> ... >>>>> I see concurrently too few tests on lower-end p2, p3 as to whether >>>>> pkg can run with lesser memory machines (routers...) (pfsense) >>>>> ... >>>>> I suspect stalling of successful frontends to bsd (pc-bsd, ghostbsd, >>>>> pfsense..) due to less-reliability, more-possibility of bugs.. >>>>> >>>> This is of some concern to me as well. A number of our utilities / >>>> scripts rely on checking /var/db/pkg as a means to test if a particular >>>> package is installed. This is often much faster than running the pkg_* >>>> commands, especially when we may be checking thousands of packages in a >>>> single run. It will be some work to adjust our utilities to using the >>>> various "pkg" commands now, but it can be done. What worries me is >>>> performance. If this is significantly slower, it may cause some issues >>>> on our end. >>> Guys, please test it before you say anything. Otherwise it's going to >>> be moved forward without you. >>> >>> >> Well, it was about time I got to doing a benchmark of this anyway :) >> >> I did quick benchmark of how one of our utilities parses through a list >> of 1k packages on a newer i5 system: >> >> First test, using /var/db/pkg/<pkg> check we have been doing: >> >> 0.178s 0:00.31 54.8% >> 0.123s 0:00.26 61.5% >> 0.099s 0:00.15 60.0% >> >> Second test, using "pkg info <pkg>": >> >> 5.347s 0:11.41 91.7% >> 5.444s 0:11.52 91.3% >> 5.878s 0:11.32 91.4% >> >> The pkg info command is quite a bit slower in this case, but 5 seconds >> isn't horrible. >> >> Now I ran the same benchmark on a slower 1.66gz Atom system, checking >> about 1200~ packages: >> >> First test, using /var/db/pkg/<pkg> check we have been doing: >> >> 0.604s 0:00.76 86.8% >> 0.622s 0:00.77 84.4% >> 0.614s 0:00.73 90.4% >> >> Second test, using "pkg info <pkg>": >> >> 28.507s 0:54.80 99.1% >> 28.282s 0:54.60 99.4% >> 28.302s 0:54.52 99.4% >> >> Now this is what concerns me a bit. It took closer to 30 seconds, which >> is quite a while to wait, especially if a utility like ours has to run >> these checks when it starts up, to show the user whats installed / not >> installed on the system. >> >> The only way around It I've found is to do a quick "pkg info" on the >> entire DB, dump that to a list, then begin to grep through that list for >> each item, but it still takes 10~ seconds on the atom. That may be what >> I end up having to do, but it still stinks to go from a half a second >> startup, to 10 seconds each time. Any other ideas on how to do this >> faster with the new pkgng? >> > Hi Kris, > > can you describe what exactly the script is doing. > > Are you aware that you can feed direct SQL to pkg ? > $> echo 'select origin,name,version,comment from packages;' | pkg shell > > At the beginning I was also a little skeptic, but even for older (slow) machines it works well here. > One note, on small systems keep an eye on /var/cache/pkg > > -- > Regards, > olli > _______________________________________________ > freebsd-current_at_freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org" I'm discussing it with Bapt off-list, but the problem isn't the initial getting of a list of packages. It's that we have to pass multiple "sets" of packages, and get some result back, like "ALL installed", "NONE installed", or "SOME installed". This means we are having to do some extra parsing after getting the initial list back, to determine those results. We have to do that with a number of collections of arbitrary packages (30+) and it is somewhat time-critical for the impatient user clicking between jails to see what package collections are installed on each :) The old way was pretty quick, usually <second even on modest systems, but I'm hopeful we can come up with some alternative method to doing the same thing quickly. -- Kris Moore PC-BSD Software iXsystemsReceived on Thu Aug 23 2012 - 18:50:28 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:30 UTC