[OE-core] [PATCH] kernel.bbclass: Fix do_shared_workdir task ordering

[Bruce Ashfield]

On Tue, Aug 11, 2015 at 10:25 AM, Stefan Müller-Klieser
<s.mueller-klieser at phytec.de> wrote:
> On 11.08.2015 15:11, Bruce Ashfield wrote:
>>
>> On Tue, Aug 11, 2015 at 9:06 AM, Bruce Ashfield
>> <bruce.ashfield at gmail.com> wrote:
>>>
>>> On Tue, Aug 11, 2015 at 8:47 AM, Otavio Salvador
>>> <otavio.salvador at ossystems.com.br> wrote:
>>>>
>>>> On Tue, Aug 11, 2015 at 12:23 AM, Bruce Ashfield
>>>> <bruce.ashfield at gmail.com> wrote:
>>>>>
>>>>> On Mon, Aug 10, 2015 at 11:21 AM, Stefan Müller-Klieser
>>>>> <s.mueller-klieser at phytec.de> wrote:
>>>>>>
>>>>>> commit 02d0a003d60326 [kernel.bbclass: Fix race condition] has
>>>>>> surfaced
>>>>>> a bug in the generation of the shared_workdir. The task
>>>>>> do_compile_kernelmodules adds the exported symbols of the kernel
>>>>>> modules
>>>>>> to the Module.symvers. By creating the shared_workdir before the
>>>>>> modules
>>>>>> are compiled, the symbols of the modules are missing in the
>>>>>> shared_workdir. Subsequent external module builds will not include the
>>>>>> ABI CRC of functions exported in modules. Modprobe will fail to load
>>>>>> the
>>>>>> external module if CONFIG_MODVERSIONS is enabled.\
>>>>>
>>>>>
>>>>> Have you seen our bug:
>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=8127 ?
>>>>>
>>>>> It's new .. so probably not.
>
> I did not. Thanks.
>
>>>>>
>>>>> The significant issue with this, is that we are now forcing anyone
>>>>> that needs the
>>>>> shared workdir artifacts to build kernel modules.
>
> That's by design. The artifacts are modified by the module build.

As was the generating of shared workdir being before the module build ..
but yes, we are aware of how and why the kernel build generates those
artifacts.

>
>>>>>
>>>>> That's performance issue for many workflows.
>>>>>
>>>>> I had some changes where I was working to short cut parts of the
>>>>> process, but
>>>>> they turned out to miss a few corner cases.
>>>>>
>>>>> We need to do more thinking on this one, before we can bring in a
>>>>> change like
>>>>> this .. since avoiding that overhead is something valuable.
>
> So you are saying a fast build is more important than a correct build?
> That's quite a bold statement.

That's not what I said.

>
>>>>
>>>> I agree that performance is important but correctness seems more
>>>> valuable for me. I think the optimization can come as a subsequent
>>>> patch ...
>>>
>>>
>>> Let's disagree on this point.
>>>
>>> There's time to get this right. We have a bug to track it, so we wont'
>>> release with
>>> the active bug, and this only hits a very tiny set of users.
>>>
>>> So we are going to step back and try and fix this right.
>
> Well, if you really want to do this then there should at least be a
> module-interdepend.bbclass not using the shared workdir and depending on the
> modules build. Fido and master are broken at the moment.

There's multiple ways to consider here and not a single right way.  But if
you have an idea, send patch for review, or update the bug I referenced,
that way we can consider them as well.

I'm not saying this isn't a bug or issue, I'm saying that it hits a certain set
of builds (and not all), so fixing this in a way that doesn't break the other
design goals of the kernel build is important as well. I end up getting
the bugs, and having to fix performance issues ... so I'm sensitive to all
the use cases.

We've been back and forth on the shared artefacts generation, with a lot
of unintended side effects. Any changes in this area have to soak and be
looked at by as many eyes as possible.

RP may grab this as is, and that's also fine with me, I'm just on the record
pointing out the side effects, so when someone notices incremental builds,
or sstate, or updates taking longer .. we can point to the reason why.

Bruce

>
> Regards,
>
> Stefan
>
>
>>
>> I hit send too soon. I have a suggestion in the bug already, so it isn't
>> like
>> we are talking about letting this sit for weeks.
>>
>> History shows that we are very unlikely to loop back and fix the
>> performance
>> of perf or other builds once the change goes in. So in the absence of
>> other concrete suggestions, looking into some other small changes is a
>> good
>> use of time.
>>
>> Cheers,
>>
>> Bruce
>>
>>
>>>
>>> Bruce
>>>
>>>>
>>>> --
>>>> Otavio Salvador                             O.S. Systems
>>>> http://www.ossystems.com.br        http://code.ossystems.com.br
>>>> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
>>>
>>>
>>>
>>>
>>> --
>>> "Thou shalt not follow the NULL pointer, for chaos and madness await
>>> thee at its end"
>>
>>
>>
>>
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"