On Sun, Jul 14, 2013 at 10:12:19AM +0000, Teske, Devin wrote: > > On Jul 14, 2013, at 2:30 AM, Matthew Seaman wrote: > > > On 14/07/2013 06:48, Teske, Devin wrote: > >> Question: Where can I learn more about the actual format of what's in > >> the new tarballs? This is going to be important not for bsdconfig, > >> but $work (we have our own build platform; I'm going to have to > >> rewrite it from mastering PLIST files to mastering YAML MANIFEST > >> files and I want to know all the gritty details). > > > > We do need a pkg-manifest(5) man page, which can double as a > > pkg-tarball(5) page since the manifest file is where most of the > > interesting bits are. > > > > A pkg tarball is a compressed tar archive like so: > > > > lucid-nonsense:...cache/pkg/All:% tar -tvf pkg-1.1.4.txz > > -rw-r--r-- 0 root wheel 530 Jan 1 1970 +COMPACT_MANIFEST > > -rw-r--r-- 0 root wheel 6385 Jan 1 1970 +MANIFEST > > -rw-r--r-- 0 root wheel 17567 Jan 1 1970 +MTREE_DIRS > > -r--r--r-- 0 root wheel 19453 Jul 7 12:26 > > /usr/local/etc/bash_completion.d/_pkg.bash > > -r-xr-xr-x 0 root wheel 629 Jul 7 12:26 > > /usr/local/etc/periodic/daily/400.status-pkg > > -r-xr-xr-x 0 root wheel 823 Jul 7 12:26 > > /usr/local/etc/periodic/daily/411.pkg-backup > > -r-xr-xr-x 0 root wheel 678 Jul 7 12:26 > > /usr/local/etc/periodic/daily/490.status-pkg-changes > > -r-xr-xr-x 0 root wheel 2558 Jul 7 12:26 > > /usr/local/etc/periodic/security/410.pkg-audit > > -r-xr-xr-x 0 root wheel 611 Jul 7 12:26 > > /usr/local/etc/periodic/security/460.pkg-checksum > > -r--r--r-- 0 root wheel 839 Jul 7 12:26 > > /usr/local/etc/pkg.conf.sample > > -r--r--r-- 0 root wheel 43432 Jul 7 12:26 /usr/local/include/pkg.h > > -r--r--r-- 0 root wheel 727558 Jul 7 12:26 /usr/local/lib/libpkg.a > > lrwxr-xr-x 0 root wheel 0 Jul 7 12:26 /usr/local/lib/libpkg.so > > -> libpkg.so.1 > > -r--r--r-- 0 root wheel 1227064 Jul 7 12:26 /usr/local/lib/libpkg.so.1 > > -rw-r--r-- 0 root wheel 187 Jul 7 12:26 > > /usr/local/libdata/pkgconfig/pkg.pc > > [... etc ...] > > > > Interesting. I notice that (while looking ahead to see a prefix: of /usr/local in the +MANIFEST), the tarball itself has files that include /usr/local in their path. > > Does pkg handle packages where the prefix (in +MANIFEST) is "/usr/local" _but_ "tar tf" of the unxz'd tarball is _not_ "/usr/local"? (e.g. pkg_install style) or is this the new way going forward? > > > > > There must at least be a +MANIFEST -- other meta data files are > > optional. +COMPACT_MANIFEST is a subset of the full +MANIFEST. They're > > both YAML documents. +COMPACT_MANIFEST looks like this, for example: > > > > --- > > name: pkg > > version: 1.1.4 > > origin: ports-mgmt/pkg > > comment: New generation package manager > > arch: freebsd:9:x86:64 > > www: https://urldefense.proofpoint.com/v1/url?u=http://wiki.freebsd.org/pkgng&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=Mrjs6vR4%2Faj2Ns9%2FssHJjg%3D%3D%0A&m=LPoff3ddLDoHE30AU6R5vgBklqxsjVAosXGjJop6uO4%3D%0A&s=30b27cf33d7e594144fc38f20bc75d84464788e715f7fc82b2495f813fd4be2d > > maintainer: portmgr_at_FreeBSD.org > > prefix: /usr/local > > licenselogic: single > > licenses: > > - BSD > > flatsize: 6311507 > > desc: | > > New Generation package management tool for FreeBSD > > > > WWW: http://wiki.freebsd.org/pkgng > > categories: > > - ports-mgmt > > shlibs_required: > > - libpkg.so.1 > > shlibs_provided: > > - libpkg.so.1 > > message: | > > If you are upgrading from the old package format, first run: > > > > # pkg2ng > > > > Nice. Very nice. > > > > > +MTREE_DIRS is a compatibility thing with the old pkg_tools. It's not > > needed in general as +MANIFEST can provide all that meta data itself. > > It isn't going to be deprecated for at least as long as the ports tree > > continues to support pkg_tools though. > > > > Beyond that, the rest of the pkg tarball just contains a tar archive of > > all the files, directories, sym-links etc to be installed by the > > package. Note that pkg(8) has no problem with creating an empty > > directory for a package, unlike pkg_tools. > > > > Excellent! > > > > Now, while you can grovel through the details of pkg tarballs and > > internal data formats like this, be aware: the format of the manifest > > and the details of the meta-data included in the pkg-tarballs is subject > > to change without warning as we develop pkg(8) further. We only promise > > API stability through the pkg(8) commands or for the libpkg.so library > > functions; reports of breakage from usage outside those APIs will > > receive little sympathy. > > > > Understood. > > So just to give you a better idea of how we manage packages here. > > We've realized that if you want to package for FreeBSD (in 9 and older), you could get by alright if you knew how to create a +CONTENTS file from scratch. For RedHat, it's the SPECFILE. And now for FreeBSD 10, it looks an awful lot like a SPECFILE (they are both in-fact YAML). The only thing you need if you really want to package is a small subset of the manifest written in YAML: Have a look at what the ports tree do to create the minimalistic manifest and what form it has and you will see how easy it is. It is so awful and complicated that there are already a load of custom scripts to create packages with pkg without using the ports tree. As an example: https://github.com/z0nt/pkg , but there is way more. I for example wrote in the past a plugin for pkg to be able to parse and USE RPM's specfiles as an input for pkg. > > So rather than teach the people here how to use tools, I taught them what I think is more important... how to manage +CONTENTS, SPECFILE, and now up-and-coming, +MANIFEST. The problem is that you compare 2 things that cannot be compared, SPECFILE is the equivalent of pkg + the ports tree. In fact specfiles are the equivalent of a given "port" !!! Are you writing rpm files directly by hand? there is no SPECFILE inside RPM but rather a header container metadata in binary format and a body which is a cpio.(gz,bz2,xz). You are not creating the RPM file directly but rather create a SPECFILE, the equivalent on FreeBSD is to create a port!!! the +MANIFEST as used to be the +CONTENTS are the equivalent of the RPM metadata. We wrote them in YAML so that they are easy to debug and that we can use third party well used and tested parsers instead of writting our own. > > However, I'd be lying if I said I taught them *all* the gory details. In reality, for +CONTENTS they edit a PLIST which is essentially +CONTENTS without the "_at_comment MD5:" entries. For SPECFILE, they manage the full thing except there's a small section that is the standard RPM bootstrap that is labeled as "do not touch". > > From what you posted of +COMPACT_MANIFEST was nice, except it lacks the list of files. > > The basic principle here is that if you have to maintain a list of files, and that list ends up being compiled into a +MANIFEST, why not just teach your peers how to read/write/maintain +MANIFEST files (in version control of course) so that there are never any mysteries as to how the package performs. Have a closer look at pkg create, you will see that we are still able to use the plist or +CONTENTS as a possible input for pkg create, it is even the main way to create packages. Once again, the main and most important requirement to build packages was to be compatible with the ports tree, and the ports tree is using plist/+CONTENTS. and pkg2ng does only works by using the +CONTENTS. > > I know this sounds screwy... but (as with our other YAML files), it's really nice because (as is the case with SPECFILE), we version-control things as-close to the end-product as possible (take for example the case of "pre-install" or "post-install" et cetera scripts -- I'm sure a tool like "pkg create" or "pkg_create" or other could do a good job of assembling the pieces into the +MANIFEST, but then it's not being version-controlled as closely. You can keep what you have already done. Once again pkg does not have the equivalent of the SPECFILE because we have the ports tree! so we do not need to provide a custom package building system. We could, as it is really easy, but we do not need so it is not there yet, as said earlier I already wrote a pkg plugin that is able to build packages from a RPM specfile, I also modified archlinux's makepkg to be able to output pkgng files. I did the same on slackare, etc. Here the limitation is not technical at all. Bapt
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:39 UTC