<div dir="ltr"><span style="color:rgb(80,0,80)">"qmake -install qinstall" reproducer from the bugzilla ticket still reproduces the issues every time even with latest pseudo, but that might be different root cause than glibc-locale issue.</span><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Aug 14, 2019 at 6:02 PM Randy MacLeod <<a href="mailto:randy.macleod@windriver.com">randy.macleod@windriver.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 8/6/19 2:51 AM, Martin Jansa wrote:<br>
> This is the same reproducer I am using in:<br>
> <a href="https://bugzilla.yoctoproject.org/show_bug.cgi?id=12434" rel="noreferrer" target="_blank">https://bugzilla.yoctoproject.org/show_bug.cgi?id=12434</a><br>
> but with this SRCREV I haven't reproduced it yet in first 500 <br>
> iterations, so it's definitely improving for me (used to reproduce it at <br>
> least once in first 500 iterations)<br>
> <br>
> Now I'm testing the reproducer with "qmake -install qinstall".<br>
<br>
Any update Martin?<br>
<br>
<br>
<br>
Using a variation of Juro's script and adding a little stress-ng load,<br>
it _seems_ that I can make the problem happen more quickly than without<br>
system stress but it's a shared system so _seems_ is underlined.<br>
<br>
Using stress-ng was supposed to be a quick check to see if I could<br>
get the reproducer down to minutes rather than around an hour.<br>
<br>
Results are promising so I'll continue to use this approach as<br>
I add debugging to pseudo and add an inline, immediate check in<br>
the context of:<br>
<br>
<a href="http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-core/glibc/glibc-locale.inc?h=master#n72" rel="noreferrer" target="_blank">http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-core/glibc/glibc-locale.inc?h=master#n72</a><br>
to see if the UID/GID are equal to my UID/GID.<br>
<br>
Test runs summaries are below.<br>
<br>
../Randy<br>
<br>
<br>
<br>
cat src/distro/yocto/b/uid-diff/glibc-locale-stress<br>
#!/bin/bash<br>
<br>
fname='glibc-locale_master_august13'<br>
max=100<br>
for (( i=1; i <= $max; i++ ))<br>
do<br>
echo "$i/$max ${fname}_$i.log"<br>
bitbake glibc-locale -c cleanall 2>&1 > /dev/null<br>
# add some stress<br>
stress-ng -t 1000 --switch 8 --switch-freq 50000 &<br>
bitbake glibc-locale 2>&1 > ${fname}_$i.log<br>
# Destress<br>
killall -9 stress-ng<br>
if grep -q "host-user-contaminated" ${fname}_$i.log; then<br>
echo "error !"<br>
exit 2<br>
#else<br>
#rm ${fname}_$i.log<br>
fi<br>
done<br>
<br>
<br>
On a (shared) system where lscpu shows 128 cores<br>
and no stress:<br>
<br>
Trial Iteration Error<br>
1 44<br>
2 19<br>
<br>
<br>
stress-ng -t 1000 --switch 8 --switch-freq 50000<br>
<br>
50000 was just the frequency that generated a high enough<br>
but not too high load. On this systems, each process used ~30% of a cpu.<br>
<br>
Trial Iteration Error<br>
1 3<br>
2 18<br>
<br>
<br>
stress-ng -t 1000 --switch 16 --switch-freq 50000<br>
<br>
Trial Iteration Error<br>
1 3<br>
2 1<br>
3 11<br>
<br>
stress-ng -t 1000 --switch 32 --switch-freq 50000<br>
<br>
Trial Iteration Error<br>
1 2<br>
2 9<br>
3 8<br>
<br>
<br>
stress-ng -t 1000 --switch 64 --switch-freq 50000<br>
<br>
Trial Iteration Error<br>
1 4<br>
2 13<br>
3 >6<br>
<br>
<br>
stress-ng -t 1000 --mq 64<br>
128 processes using 98% cpu each<br>
<br>
Trial Iteration Error<br>
1 14<br>
2 NaN<br>
<br>
Trial 2 was precluded by other users of the shared system complaining!<br>
The idea was to cause more rapid context switches. Later, I might try<br>
this again with say 16 workers. If anyone has a better idea, please<br>
reply.<br>
<br>
EOM<br>
<br>
> <br>
> Regards,<br>
> <br>
> On Tue, Aug 6, 2019 at 12:43 AM Bystricky, Juro <br>
> <<a href="mailto:juro.bystricky@intel.com" target="_blank">juro.bystricky@intel.com</a> <mailto:<a href="mailto:juro.bystricky@intel.com" target="_blank">juro.bystricky@intel.com</a>>> wrote:<br>
> <br>
> I can reproduce the problem fairly easily (and, sadly even with the<br>
> latest commits as 060058bb29f70b244e685b3c704eb0641b736f73 ).<br>
> In my case, it seems easy to reproduce if I have 40+ threads running.<br>
> The reproducer script (below) fails typically within the first 10<br>
> iterations.<br>
> <br>
> <br>
> #!/bin/bash<br>
> <br>
> fname='glibc-locale_master_august8'<br>
> max=1000<br>
> for (( i=1; i <= $max; i++ ))<br>
> do<br>
> echo "$i/$max ${fname}_$i.log"<br>
> bitbake glibc-locale -c cleanall 2>&1 > /dev/null<br>
> bitbake glibc-locale 2>&1 > ${fname}_$i.log<br>
> if grep -q "host-user-contaminated" ${fname}_$i.log; then<br>
> echo "error !"<br>
> exit 2<br>
> #else<br>
> #rm ${fname}_$i.log<br>
> fi<br>
> <br>
> done<br>
> <br>
> ________________________________________<br>
> From: <a href="mailto:openembedded-core-bounces@lists.openembedded.org" target="_blank">openembedded-core-bounces@lists.openembedded.org</a><br>
> <mailto:<a href="mailto:openembedded-core-bounces@lists.openembedded.org" target="_blank">openembedded-core-bounces@lists.openembedded.org</a>><br>
> [<a href="mailto:openembedded-core-bounces@lists.openembedded.org" target="_blank">openembedded-core-bounces@lists.openembedded.org</a><br>
> <mailto:<a href="mailto:openembedded-core-bounces@lists.openembedded.org" target="_blank">openembedded-core-bounces@lists.openembedded.org</a>>] on behalf<br>
> of Seebs [<a href="mailto:seebs@seebs.net" target="_blank">seebs@seebs.net</a> <mailto:<a href="mailto:seebs@seebs.net" target="_blank">seebs@seebs.net</a>>]<br>
> Sent: Saturday, August 03, 2019 7:23 AM<br>
> To: Khem Raj<br>
> Cc: <a href="mailto:openembedded-core@lists.openembedded.org" target="_blank">openembedded-core@lists.openembedded.org</a><br>
> <mailto:<a href="mailto:openembedded-core@lists.openembedded.org" target="_blank">openembedded-core@lists.openembedded.org</a>><br>
> Subject: Re: [OE-core] [PATCH v2] pseudo: Upgrade to latest to fix<br>
> openat() with a directory symlink [NAK]<br>
> <br>
> On Sat, 3 Aug 2019 05:33:46 -0700<br>
> Khem Raj <<a href="mailto:raj.khem@gmail.com" target="_blank">raj.khem@gmail.com</a> <mailto:<a href="mailto:raj.khem@gmail.com" target="_blank">raj.khem@gmail.com</a>>> wrote:<br>
> <br>
> > Will this fix the file ownership issue that we see with Glibc-locale<br>
> > packages from time to time?<br>
> <br>
> I have no idea. Since I haven't got a reliable reproducer for it, I<br>
> can't test it in a sane way.<br>
> <br>
> -s<br>
> --<br>
> _______________________________________________<br>
> Openembedded-core mailing list<br>
> <a href="mailto:Openembedded-core@lists.openembedded.org" target="_blank">Openembedded-core@lists.openembedded.org</a><br>
> <mailto:<a href="mailto:Openembedded-core@lists.openembedded.org" target="_blank">Openembedded-core@lists.openembedded.org</a>><br>
> <a href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" rel="noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/openembedded-core</a><br>
> -- <br>
> _______________________________________________<br>
> Openembedded-core mailing list<br>
> <a href="mailto:Openembedded-core@lists.openembedded.org" target="_blank">Openembedded-core@lists.openembedded.org</a><br>
> <mailto:<a href="mailto:Openembedded-core@lists.openembedded.org" target="_blank">Openembedded-core@lists.openembedded.org</a>><br>
> <a href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" rel="noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/openembedded-core</a><br>
> <br>
> <br>
<br>
<br>
-- <br>
# Randy MacLeod<br>
# Wind River Linux<br>
</blockquote></div>