[OE-core] [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

Thu Aug 2 20:55:30 UTC 2018

On 08/02/2018 01:53 AM, Richard Purdie wrote:
> On Thu, 2018-08-02 at 10:55 +0800, Anuj Mittal wrote:
>> On 08/02/2018 07:30 AM, Alejandro Enedino Hernandez Samaniego wrote:
>>> Hey Anuj,
>>>
>>>
>>> On 08/01/2018 04:25 AM, Anuj Mittal wrote:
>>>> On 07/31/2018 05:21 AM, Jaewon Lee wrote:
>>>>> When using a recipe space kernel-meta, scc files are added
>>>>> through
>>>>> SRC_URI, but they may include corresponding kernel fragments
>>>>> that are
>>>>> not necessarily in SRC_URI.
>>>>>
>>>>> For bitbake, this is not a problem because the kernel-yocto
>>>>> class adds
>>>>> the path where the .scc file was found to includes which
>>>>> consequentially
>>>>> makes the .cfg file available to the kernel build.
>>>>>
>>>>> However, when using devtool, only files specified in SRC_URI
>>>>> are copied
>>>>> to oe-local-files in devtool's workspace. So if the cfg file is
>>>>> not in
>>>>> SRC_URI, it won't be copied, causing a kernel build failure
>>>>> when trying
>>>>> to find it.
>>>>>
>>>>> This fix parses local .scc files in SRC_URI, copies the
>>>>> corresponding .cfg
>>>>> file to devtool's workdir, and also adds it to local_files so
>>>>> it is
>>>>> available when doing a devtool build for the kernel.
>>>>>
>>>>> [YOCTO #12858]
>>>>>
>>>>> Signed-off-by: Jaewon Lee <jaewon.lee at xilinx.com>
>>>>> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@
>>>>> xilinx.com>
>>>>> ---
>>>>>    meta/classes/devtool-source.bbclass | 12 ++++++++++++
>>>>>    1 file changed, 12 insertions(+)
>>>>>
>>>>> diff --git a/meta/classes/devtool-source.bbclass
>>>>> b/meta/classes/devtool-source.bbclass
>>>>> index 56882a4..c70fea2 100644
>>>>> --- a/meta/classes/devtool-source.bbclass
>>>>> +++ b/meta/classes/devtool-source.bbclass
>>>>> @@ -90,11 +90,23 @@ python devtool_post_unpack() {
>>>>>                            fname in files])
>>>>>            return ret
>>>>>    
>>>>> +    is_kernel_yocto = bb.data.inherits_class('kernel-yocto',
>>>>> d)
>>>>>        # Move local source files into separate subdir
>>>>>        recipe_patches = [os.path.basename(patch) for patch in
>>>>>                            oe.recipeutils.get_recipe_patches(d)]
>>>>>        local_files = oe.recipeutils.get_recipe_local_files(d)
>>>>>    
>>>>> +    if is_kernel_yocto:
>>>>> +      for key in local_files.copy():
>>>>> +        if key.endswith('scc'):
>>>>> +          sccfile = open(local_files[key], 'r')
>>>>> +          for l in sccfile:
>>>>> +            line = l.split()
>>>>> +            if line and line[0] == 'kconf' and line[-
>>>>> 1].endswith('.cfg'):
>>>>> +              local_files[line[-1]] =
>>>>> os.path.join(os.path.dirname(local_files[key]), line[-1])
>>>>> +              shutil.copy2(os.path.join(os.path.dirname(local_
>>>>> files[key]), line[-1]), workdir)
>>>>> +          sccfile.close()
>>>>> +
>>>> Would the patches included in these .scc files also need to be
>>>> handled
>>>> in the same way? Would this also work if there are other scc
>>>> files
>>>> included in a scc file?
>>> Yes, I believe that the same mechanism should be used for patch
>>> files as
>>> well,
>>> basically anything that may be needed to build but that its not
>>> necessarily
>>> explicitly listed on SRC_URI.
>>>
>>> I believe it will work for other scc files and it doesnt have to
>>> be
>>> recursive,
>>> because while cfg files arent required to be in SRC_URI , scc files
>>> ARE
>>> required
>>> to be SRC_URI, which means that this will eventually find the other
>>> scc
>>> file on the list
>> I don't think they are required to be specified except for the top
>> level
>> one. At least, when I test it, I see problems. kernel-tools spp
>> script
>> parses them recursively and looks for a nested scc even if it is not
>> specified as part of SRC_URI. That is how the top level sccs from
>> kernel-cache are parsed too. Can you give it a try please?
Hey Anuj,
We can give it a try but do you have a specific example of how it fails 
for you?
Because I know theres lots of nested sccs on the yocto kernel cache, but 
in that case
it wouldn't be a problem since this bug is specifically introduced by 
devtool when it
copies local files from SRC_URI to a oe-local-files directory, it misses 
the corresponding cfg files (or patch files)
since they're not listed in SRC_URI and it fails to build, in the case 
of the yocto kernel cache, it
does not contain local files, so they wont be moved hence why it 
wouldn't be an issue.

 From the kernel dev manual:
https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#recipe-space-metadata 
:
It is only necessary to specify the|.scc|files on the|SRC_URI| 
<http://www.yoctoproject.org/docs/2.5/ref-manual/ref-manual.html#var-SRC_URI>.

It specifies that only the scc files need to be on SRC_URI, which is why 
I'm saying the script
will eventually run into them from the list.

I want to be clear that I'm not against doing this recursively, I just 
want to understand your testcase better.
> What would be really great here would be some test cases in the qa
> framework! It'd be a good way to both illustrate the problems and then
> test we've fixed it and that it stays fixed.
Hey Richard,
Sure that sounds like a good idea, well work on adding a testcase (or 
adding to the existing one) for devtool when
it is using a recipe-space kernel -meta.

Alejandro
>
> Cheers,
>
> Richard
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20180802/598a1c33/attachment-0002.html>