Re: FreeBSD CI Weekly Report 2020-04-12

On 15 Apr 2020, at 0:37, Li-Wen Hsu wrote:
> (Please send the followup to freebsd-testing_at_ and note Reply-To is 
> set.)
>
> FreeBSD CI Weekly Report 2020-04-12
> ===================================
>
> Here is a summary of the FreeBSD Continuous Integration results for 
> the period
> from 2020-04-06 to 2020-04-12.
>
> During this period, we have:
>
> * 1801 builds (94.0% (+0.4) passed, 6.0% (-0.4) failed) of buildworld 
> and
>   buildkernel (GENERIC and LINT) were executed on aarch64, amd64, 
> armv6,
>   armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64,
>   sparc64 architectures for head, stable/12, stable/11 branches.
> * 288 test runs (25.1% (-24.6) passed, 29.9% (+10.6) unstable, 45.1% 
> (+14.1)
>   exception) were executed on amd64, i386, riscv64 architectures for 
> head,
>   stable/12, stable/11 branches.
> * 30 doc and www builds (83.3% (-1.3) passed, 16.7% (+1.3) failed)
>
> Test case status (on 2020-04-12 23:59):
> | Branch/Architecture | Total     | Pass       | Fail     | Skipped  |
> | ------------------- | --------- | ---------- | -------- | -------- |
> | head/amd64          | 7744 (+4) | 7638 (+19) | 14 (+5)  | 92 (-20) |
> | head/i386           | 7742 (+4) | 7628 (+15) | 16 (+5)  | 98 (-16) |
> | 12-STABLE/amd64     | 7508 (0)  | 7449 (-3)  | 1 (+1)   | 58 (+2)  |
> | 12-STABLE/i386      | 7506 (0)  | 7425 (-17) | 2 (+2)   | 79 (+15) |
> | 11-STABLE/amd64     | 6882 (0)  | 6829 (-6)  | 1 (+1)   | 52 (+5)  |
> | 11-STABLE/i386      | 6880 (0)  | 6749 (-82) | 80 (+80) | 51 (+2)  |
>
> (The statistics from experimental jobs are omitted)
>
> If any of the issues found by CI are in your area of interest or 
> expertise
> please investigate the PRs listed below.
>
> The latest web version of this report is available at
> https://hackmd.io/_at_FreeBSD-CI/report-20200412 and archive is available 
> at
> https://hackmd.io/_at_FreeBSD-CI/ , any help is welcome.
>
> ## News
>
> * The test env now loads the required module for firewall tests.
>
> * New armv7 job is added (to replace armv6 one):
>   * FreeBSD-head-armv7-testvm
>   The images are available at https://artifact.ci.freebsd.org
>   FreeBSD-head-armv7-test is ready but needs test env update.
>
> ## Failing jobs
>
> * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/
>   * See console log for the error details.
>
> ## Failing tests
>
> * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/
>   * local.kyua.integration.cmd_about_test.topic__authors__installed
>   * sys.netipsec.tunnel.empty.v4
>   * sys.netipsec.tunnel.empty.v6
>   * sys.netpfil.common.forward.ipf_v4
>   * sys.netpfil.common.forward.ipfw_v4
>   * sys.netpfil.common.forward.pf_v4
>   * sys.netpfil.common.tos.ipfw_tos
>   * sys.netpfil.common.tos.pf_tos
>   * sys.netpfil.pf.forward.v4
I can’t actually reproduce this failure in my test VM, but with the ci 
test VM I can reproduce the problem.
It’s not related to pf, because the sanity check ping we do before we 
set up pf already fails.
Or rather pft_ping.py sends an incorrect packet, because `ping` does get 
the packet to go where it’s supposed to go.

Scapy seems to fail to find the source IP address, so we get this:

	12:12:22.152652 IP 0.0.0.0 > 198.51.100.3: ICMP echo request, id 0, seq 
0, length 12

I have a vague recollection that we’ve seem this problem before, but I 
can’t remember what we did about it.

In all likelihood most of the other netpfil tests fail for exactly the 
same reason.

Best regards,
Kristof