Re: FYI: WITH_REPRODUCIBLE_BUILD= problem for some files?

On 2021-May-3, at 21:27, Mark Millard <marklmi at yahoo.com> wrote:

> On 2021-May-3, at 19:26, Mark Millard <marklmi at yahoo.com> wrote:
> 
>> On 2021-May-3, at 10:51, Mark Millard <marklmi at yahoo.com> wrote:
>> 
>>> On 2021-May-3, at 07:47, Ed Maste <emaste at freebsd.org> wrote:
>>> 
>>>> On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current
>>>> <freebsd-current_at_freebsd.org> wrote:
>>>>> 
>>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ
>>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and /usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ
>>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and /usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ...
>>>> 
>>>> This is unexpected. Unfortunately I haven't looked at reproducibility
>>>> in a while, and my work was all on x86. This could be a regression or
>>>> a longstanding issue with arm64.
>>>> 
>>>> If you install the diffoscope package (py37-diffoscope) and run it on
>>>> the two directories / files it should give a more convenient view of
>>>> the differences. (Or, if you can make a tarball of the differing files
>>>> I can take a look.)
>>> 
>>> I no longer have the same content in those directory
>>> trees: newer rebuild and the same buildworld used to
>>> installworld to both places, instead of 2 different
>>> buildworld's. I'm also unsure how reproducible getting
>>> differences was.
>>> 
>>> I can eventually do experiments to test multiple separate
>>> buildworld's and installworld's, but the machine is busy
>>> building ports and the llvm builds involved means it
>>> will be some time before I'd switch activities. And the
>>> buildworld's involve llvm builds as well and take notable
>>> time themselves. So my next comparison will not be any
>>> time soon.
>>> 
>>> I'll let you know if I manage to generate another example,
>>> this time being sure to keep the data. If I try multiple
>>> times without finding any differences, I'll eventually
>>> decide "enough is enough" and let you know.
>> 
>> I've still got a long ways to go to do the first
>> actual comparison of builds.
>> 
>> But I'll note that I've built and stalled py37-diffoscope
>> (new to me). A basic quick test showed that it reports:
>> 
>> W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" module is unavailable.
>> 
>> As I'm not familiar with the tool, you might need to send
>> notes about how you want me to use the tool to get the
>> output that you would want. (And, so, I get to learn . . .)
> 
> I've tried another experiment (* in the path matches "28" and "30"):
> 
> # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh
> $<3/>2021-05-03 21:08:48 W: diffoscope.main: Fuzzy-matching is currently disabled as the "tlsh" module is unavailable.
> $<3/>Traceback (most recent call last):
>  File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line 745, in main
>    sys.exit(run_diffoscope(parsed_args))
>  File "/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line 677, in run_diffoscope
>    difference = load_diff_from_path(path1)
>  File "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", line 31, in load_diff_from_path
>    return load_diff(codecs.getreader("utf-8")(fp), path)
>  File "/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py", line 35, in load_diff
>    return JSONReaderV1().load(fp, path)
>  File "/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py", line 33, in load
>    raw = json.load(fp)
>  File "/usr/local/lib/python3.7/json/__init__.py", line 293, in load
>    return loads(fp.read(),
>  File "/usr/local/lib/python3.7/codecs.py", line 504, in read
>    newchars, decodedbytes = self.decode(data, self.errors)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 18: invalid start byte
> 
> The two older snapshots of a Boot Environment have
> bin/sh files that compare equal. But every program I
> tried the above sort of thing against on got the same
> UnicodeDecodeError result from diffoscope, byte value
> and position matching.
> 
> These snapshots have more than an installworld in them
> and so are messy to compare overall. But the
> installworld (and installkernel) content show similar
> differences to what I reported before as far as
> example files with differences go. But this is aarch64,
> not armv7.
> 
> It will still be notable time before I have simple
> installworld tree's to compare.

While waiting for the 2nd buildworld to complete, I used
the 1st buildworld's materials to installworld twice (to
different, empty directory trees) and then did a diff.
The diff reported:

# diff -rq /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/ /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm2/ | more
diff: /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local: No such file or directory
diff: /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/sys/pjdfstest/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests: Too many levels of symbolic links

FYI:

# ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local
lrwxr-xr-x  1 root  wheel  14 May  3 23:18:22 2021 /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local -> ../local/tests

So this looks like the expected result and suggests that
the problem(s) are at the buildworld stage. (No surprise.)

I should be sleeping by the time the 2nd buildworld for
cortex-a72 (aarch64) is done. (That is what I decided
to test initially.) So it will be more hours until I have
basic comparison results from 2 distinct buildworlds.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)