Ordering of files in zoneinfo [Was Re: sort.core error doing installworld on Current.]

From: Xin LI <delphij_at_gmail.com>
Date: Thu, 23 Apr 2020 09:44:59 -0700
Hi,

Thanks for raising this.

I have took a look at the change history, it seems that the find operation
was introduced in r245265
<https://svnweb.freebsd.org/base?view=revision&revision=245265> (brooks_at_,
to support packaged base) and sort was initially implemented as find -s in
r289451 <https://svnweb.freebsd.org/base?view=revision&revision=289451>
(ngie_at_, to make METALOG reproducible) then as sort in r328958
<https://svnweb.freebsd.org/base?view=revision&revision=328958> (imp_at_, for
portability).

I wonder if we could drop the sort and replace ${TZS} in line 100 with
${TZS:O} instead?

By the way, looking at
https://github.com/freebsd/pkg/blob/master/libpkg/metalog.c , I wonder if
the sort should really happen in pkg(8) instead?

On Fri, Apr 17, 2020 at 7:28 AM Johan Hendriks <joh.hendriks_at_gmail.com>
wrote:

> Op 17-04-2020 om 13:30 schreef Johan Hendriks:
> >
> > Op 17-04-2020 om 12:47 schreef Rodney W. Grimes:
> >>>> Op 17-04-2020 om 03:31 schreef Rodney W. Grimes:
> >>>>>> On Thu, Apr 16, 2020 at 12:39 PM Kevin Oberman
> >>>>>> <rkoberman_at_gmail.com> wrote:
> >>>>>>
> >>>>>>> So you some how had a sort core dump sitting in
> >>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir. The
> >>>>>>> questions, how
> >>>>>>> did get there? I'd take a look at the date on the file and, it
> >>>>>>> it is older
> >>>>>>> than the buildworld, just assume that it was left-over garbage.
> >>>>>>> In either
> >>>>>>> case, you can delete it and do another installworld.
> >>>>>>>
> >>>>>>> That should most likely fix things, but, if the buildworld or
> >>>>>>> installworld
> >>>>>>> had a crash of sort(1) that left the file, further investigation
> >>>>>>> might be
> >>>>>>> needed. Re-making the zoneinfo would help track it down should
> >>>>>>> this be a re
> >>>>>>> al bug, but it's my uneducated guess that it's not.
> >>>>>>> --
> >>>>>>> Kevin Oberman, Part time kid herder and retired Network Engineer
> >>>>>>> E-mail: rkoberman_at_gmail.com
> >>>>>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
> >>>>>>>
> >>>>>> Please forgive that awful post! I missed a part of your message
> >>>>>> by laziness.
> >>>>>>
> >>>>>> It's odd that the error of sort(1) crashing was not caught by the
> >>>>>> script.
> >>>>> Yes, that is a Makefile flaw someplace.
> >>>>> Further there must be a wildcard being used to decide which files to
> >>>>> install, that is a further Makefile flaw.  Wildcards should NOT be
> >>>>> used
> >>>>> in the source of an install list, exactly because of this type of
> >>>>> cruft
> >>>>> that can be dropped in an obj dir.
> >>  From src/share/zoneinfo/Makefile at about line 93:
> >> 92  if make(*install*)
> >> 93  TZS!= cd ${TZBUILDDIR} && find * -type f | LC_ALL=C sort
> >>                                   ^^^^ this is a very bad thing to do
> >> in a Makefile.
> >>
> >> 94  .endif
> >>
> >> Now I still don't know why sort cored, but I am sure this is the line
> >> that did it.
> >>
> >>>>>> Clearly, sort should NOT crash! Again, a re-build of zoneinfo
> >>>>>> might catch
> >>>>>> something. Looking at the core might tell you which "sort" was
> >>>>>> involved...
> >>>>>> the one you just built or the one in the base system. This could
> >>>>>> be just a
> >>>>>> FOTU, but I would not bet on it.
> >>>>> I suspect a recent zoneinfo commit as the root cause.
> >>>>>
> >>>> I have no idea how to bypass this issue.
> >>>> I have used sort from the latest snapshot and placed that file on the
> >>>> system and in the build dir, but i keep getting the core
> >>>>
> >>>> How can i test an build and install part for zoneinfo
> >>>>
> >>>> If i go into the dir /usr/src/share/zoneinfo and do make install it
> >>>> does
> >>>> not work, do i need to add something?
> >>> Can you show us the output from
> >>> cd /usr/src/share/zoneinfo
> >>> make clean && make depend && make all && make install
> >>> Someplace in that we should get to see sort crashing...
> >>>
> > On both machines my src.conf file is the same.
> >
> > I will start over from a clean world by doing a make cleanworld and
> > see if it then still gives the errors
> > Maybe some old artifacts are hanging around.
> >
> >
> >
> >>>
> >>>> Thank you both for your time
> >>>>
> >>>>>> --
> >>>>>> Kevin Oberman, Part time kid herder and retired Network Engineer
> >>>>>> E-mail: rkoberman_at_gmail.com
> >>>>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
> >>>>>>
> >>>>>>
> >>>>>>> On Thu, Apr 16, 2020 at 11:49 AM Johan Hendriks
> >>>>>>> <joh.hendriks_at_gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I have a machine running FreeBSD head.
> >>>>>>>> rev 13.0-CURRENT #11 r360008
> >>>>>>>>
> >>>>>>>> This is a quite powerful machine, so i thought it was a good
> >>>>>>>> idea to let
> >>>>>>>> that server do the build and for my virtualbox machine i can
> >>>>>>>> use the
> >>>>>>>> powerful machine to do a installword over NFS.
> >>>>>>>>
> >>>>>>>> But when i did the make installworld step the client so to say
> >>>>>>>> gives an
> >>>>>>>> error.
> >>>>>>>>
> >>>>>>>> install   -o root -g wheel -m 444
> >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/Zulu
> >>>>>>>> /usr/share/zoneinfo/Zulu
> >>>>>>>> install   -o root -g wheel -m 444
> >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/posixrules
> >>>>>>>> /usr/share/zoneinfo/posixrules
> >>>>>>>> install   -o root -g wheel -m 444
> >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/sort.core
> >>>>>>>> /usr/share/zoneinfo/sort.core
> >>>>>>>> install:
> >>>>>>>> /usr/obj/usr/src/amd64.amd64/share/zoneinfo/builddir/sort.core:
> >>>>>>>> Permission denied
> >>>>>>>> *** Error code 71
> >>>>>>>>
> >>>>>>>> Stop.
> >>>>>>>> bmake[5]: stopped in /usr/src/share/zoneinfo
> >>>>>>>> *** Error code 1
> >>>>>>>>
> >>>>>>>> Stop.
> >>>>>>>> bmake[4]: stopped in /usr/src/share
> >>>>>>>> *** Error code 1
> >>>>>>>>
> >>>>>>>> Stop.
> >>>>>>>> bmake[3]: stopped in /usr/src
> >>>>>>>> *** Error code 1
> >>>>>>>>
> >>>>>>>> Stop.
> >>>>>>>> bmake[2]: stopped in /usr/src
> >>>>>>>> *** Error code 1
> >>>>>>>>
> >>>>>>>> Stop.
> >>>>>>>> bmake[1]: stopped in /usr/src
> >>>>>>>> *** Error code 1
> >>>>>>>>
> >>>>>>>> Stop.
> >>>>>>>> make: stopped in /usr/src
> >>>>>>>> .ERROR_TARGET='installworld'
> >>>>>>>> .ERROR_META_FILE=''
> >>>>>>>> .MAKE.LEVEL='0'
> >>>>>>>> MAKEFILE=''
> >>>>>>>> .MAKE.MODE='normal'
> >>>>>>>> _ERROR_CMD='.PHONY'
> >>>>>>>> .CURDIR='/usr/src'
> >>>>>>>> .MAKE='make'
> >>>>>>>> .OBJDIR='/usr/obj/usr/src/amd64.amd64'
> >>>>>>>> .TARGETS='installworld'
> >>>>>>>> DESTDIR=''
> >>>>>>>> LD_LIBRARY_PATH=''
> >>>>>>>> MACHINE='amd64'
> >>>>>>>> MACHINE_ARCH='amd64'
> >>>>>>>> MAKEOBJDIRPREFIX='/usr/obj'
> >>>>>>>> MAKESYSPATH='/usr/src/share/mk'
> >>>>>>>> MAKE_VERSION='20181221'
> >>>>>>>> PATH='/sbin:/bin:/usr/sbin:/usr/bin'
> >>>>>>>> SRCTOP='/usr/src'
> >>>>>>>> OBJTOP='/usr/obj/usr/src/amd64.amd64'
> >>>>>>>>
> >>>>>>>> It looks likes sort coredumps in the usr/share/zoneinfo part of
> >>>>>>>> the base.
> >>>>>>>> As it has no permission on the NFS share it errors out.
> >>>>>>>> On the server itself, the installworld goes well, but it leaves a
> >>>>>>>> sort.core file behind in /usr/share/zoneinfo
> >>>>>>>>
> >>>>>>>> cd /usr/share/zoneinfo
> >>>>>>>> ls -al
> >>>>>>>>
> >>>>>>>>
> >>>>>> _______________________________________________
> >>>>>> freebsd-current_at_freebsd.org mailing list
> >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> >>>>>> To unsubscribe, send any mail to
> >>>>>> "freebsd-current-unsubscribe_at_freebsd.org"
> >>>>>>
> >>>> _______________________________________________
> >>>> freebsd-current_at_freebsd.org mailing list
> >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> >>>> To unsubscribe, send any mail to
> >>>> "freebsd-current-unsubscribe_at_freebsd.org"
> >>>>
> >>> --
> >>> Rod Grimes rgrimes_at_freebsd.org
> >>> _______________________________________________
> >>> freebsd-current_at_freebsd.org mailing list
> >>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> >>> To unsubscribe, send any mail to
> >>> "freebsd-current-unsubscribe_at_freebsd.org"
> >>>
> I have rebuild everything on the host and did a make cleanworld.
> Al is fine now.
> I should have done that before i asked here.
> Sorry to have wasted your time.
> But we did find a Makefile that should be doing things differently.
>
> regards
> Johan
>
>
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>
Received on Thu Apr 23 2020 - 14:45:12 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:23 UTC