Re: Need help fixing tests in CURRENT

On Tue, Sep 9, 2014 at 11:13 AM,  <yaneurabeya_at_gmail.com> wrote:
> On Sep 9, 2014, at 10:11, Allan Jude <allanjude_at_freebsd.org> wrote:
>
>> On 2014-09-09 13:05, Craig Rodrigues wrote:
>>> Hi,
>>>
>>> I did a buildworld/installworld of current in a bhyve VM, and then did the
>>> following.
>>>
>>> pkg install devel/kyua
>>> cd /usr/tests
>>> kyua test
>>> kyua report
>>> kyua report-html
>>> kyua report-junit
>>>
>>> I then published the results in Jenkins:
>>>
>>> The results are here:
>>>
>>> https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/lastCompletedBuild/testReport/
>>>
>>> There seem to be 31 test failures.  Can folks look into those
>>> and see if the tests need to be fixed
>>>
>>> Right now I ran these steps manually, but hopefully over time
>>> we can automate more of this stuff.
>>
>> Most of the errors (jot, m4, sed, etc) seems to be:
>>
>> kyua-tap-tester: Configuration variables not supported; ignoring
>> 'has.cleanup=false'
>> kyua-tap-tester: Configuration variables not supported; ignoring
>> 'unprivileged-user=tests’
>
> Not quite…
>
>> while everything else about the test was successful.
>
> Hi Craig,
>         Taking a stab at the issues, most of the problems are because of https://github.com/jmmv/kyua/issues/103 . The other issues with atf ( https://github.com/jmmv/atf/issues/8 ) and kyua ( https://github.com/jmmv/kyua/issues/109 ) are known (I’m sure about ATF, not 100% sure about kyua, but my gut instinct says yes).
> Thanks!
> -Garrett

There are still a few genuine failures.  usr.bin.yacc.yacc_tests.main
is failing for reasons that have nothing to do with ATF or Kyua.  So
are bin.pkill.pkill-j_test.main and bin.pkill.pgrep-j_test.main.