[OE-core] Help with intermittent autobuilder exposed bugs

[akuster808]

On 10/20/19 5:54 AM, Richard Purdie wrote:
> We have a situation brewing on the autobuilder. Each run there are some
> bugs which sometimes can appear. Law of averages says one or more of
> the issues below will now appear as we have so many of them. The
> project's standard processes aren't helping to capture the issues or
> track them down. 
>
> With the release mostly sorted out from my perspective I can look at
> some of the other health aspects of the project.
>
> In theory SWAT should have filed bugs for these. In most cases they
> haven't. If they did, we'd discuss them at triage, decide they looked
> hard and then there wouldn't be anyone to "assign" to them so they'd
> most likely end up on my plate anyway. I find it hard to know what to
> do with these. I already have a load in my bug backlog which I've not
> gotten sorted out so adding more probably won't help.
>
> To summarise the issues [just from the last few builds]:

I'll get started logging bugs
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/756
> oe-selftest recipetool.RecipetoolTests.test_recipetool_load_plugin
> RP has a long standing bug:
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13070
> https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/1177
> qa-extras2 step2c testimage logrotate.LogrotateTest.test_2_logrotate
> for core-image-sato
> [no open bug?]
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13602


>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/441
> oe-selftest devtool.DevtoolExtractTests.test_devtool_deploy_target
> [no open bug, unhelpful backtrace from tinfoil]

Bug #13601, seen with stable/warrior-nmut and on same host - fedora30-ty-1



> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/440
> oe-sefltest signing.LockedSignatures.test_locked_signatures
> [no open bug or any idea why it failed]
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13605
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/440
> oe-selftest distrodata.Distrodata.test_maintainer
> [no open bug, same unhelpful backtrace from tinfoil]
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13604
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/440
> oe-selftest reproducible.ReproducibleTests.test_reproducible_builds
> [no open bug?, fedora specific perldoc reproducibility issue]
Think its the same as
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13602
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/52
> qemuarm systemd timeout parselogs error
> """Central error: Oct 19 11:21:48 qemuarm login[228]:I have 
> pam_systemd(login:session): Failed to create session: Connection timed
> out"""
> [no open bug]
I am seeing this on one of my arm machines at home when it boots so I
will take this one.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13603

>
> Some of these are with master-next but I don't believe are related to
> any issue specific to -next, its just because -next has had more test
> runs.
>
> I'm not sure what we do about this. The SWAT process really needs to
> pay more attention, equally the people are volunteers and therefore I
> try to be grateful for any help.
>
> These issues are becoming more frequent and mean we lose confidence in
> the builds which delays patch merging. People often ask me how they can
> help, ideas on how we can move forward with this would be welcome...
>
> I also know Armin has a difficult to debug selftest issue with warrior
> which is blocking its release build right now.
>
> Cheers,
>
> Richard
>