[OE-core] KERNEL_IMAGETYPES and Image.gz

[Nicolas Dechesne]

On Tue, May 30, 2017 at 11:56 PM, Andre McCurdy <armccurdy at gmail.com> wrote:
> On Tue, May 30, 2017 at 1:26 PM, Nicolas Dechesne
> <nicolas.dechesne at linaro.org> wrote:
>> On Tue, May 30, 2017 at 7:15 PM, Andre McCurdy <armccurdy at gmail.com> wrote:
>>>>   gzip -9c < "${typeformake}" > "${KERNEL_OUTPUT_DIR}/${type}"
>>>> It really sounds to me like something obvious is missing, i doubt I
>>>> would be the first one to try Image.gz..
>>>
>>> You may be the first to try for a non-MIPS machine.
>>>
>>> MIPS is a little special in that the MIPS kernel doesn't support self
>>> extracting compressed kernels (zImage, etc), so MIPS machines are the
>>> most likely to make use of a gzipped vmlinux image (which will then be
>>> decompressed by the bootloader) and KERNEL_IMAGETYPES="vmlinux.gz"
>>> works OK.
>>>
>>> If a compressed kernel is required, most non-MIPS machines use a self
>>> extracting image.
>>
>> this piece of is coming from this old commit:
>>
>> cb17b6c2a7 kernel.bbclass: support kernel image type of vmlinux.gz
>>
>> It is clear that the goal of this logic is only to support vmlinux.gz
>> since the kernel build system does not create it (only vmlinux). It
>> almost looks to me like we should treat vmlinux.gz as a special case
>> instead of treating each .gz target as 'special'
>>
>> -    typeformake = re.sub(r'\.gz', '', types)
>> +    typeformake = re.sub(r'vmlinux\.gz', '', types)
>>
>> e.g. the special case should be used *only* when the kernel build
>> system is not capable to create the compressed file. e.g. on ARM ,
>> Image, Image.gz and vmlinux are valid image that the kernel build can
>> generate, so it probably doesn't make any sense to have vmlinux.gz in
>> KERNEL_IMAGETYPES anyways..
>
> I agree that kernel.bbclass should not take over creation of an image
> type which the kernel can handle directly, and if that means special
> cases in kernel.bbclass to handle differences in which image types can
> be created by the kernel for each architecture then so be it.
> Documenting the special cases makes it less likely they will get
> overlooked and broken by future refactoring etc.
>
> I'm not sure about not making sense to have "vmlinux.gz" in
> KERNEL_IMAGETYPES though? It is a valid image type for MIPS and it
> should be possible for machine configs to choose to build it.

that's fine. i am not familiar with MIPS, if you say so, it's ok ;-)

>
>> In fact , looking more closely at kernel makefiles (for arch/mips,
>> since you brought this one up), it looks like uImage (and compressed
>> vmlinux) was added in 2013 (commit
>> 4defe4559e86e26545ab2f542656a4b966dcde53). So we might not need all
>> this logic at all anymore in kernel.bbclass and just get rid of it.
>
> I think vmlinux is an elf file and vmlinux.bin is the raw binary (?),
> so even though the kernel can create vmlinux.bin.gz directly it
> doesn't cover systems which need a gzipped elf file (ie vmlinux.gz).
>
>> so, to backtrack a bit, i think it should be safe to only treat
>> vmlinux.gz as a 'special' case, and then it is there in
>> KERNEL_IMAGETYPE, then we do a "make vmlinux" and we manually compress
>> the file, otherwise we just trust that KERNEL_IMAGETYPE is a valid
>> type and give it to kernel make command line.  that should be enough
>> to fix the Image.gz cases on all platforms where it's relevant (like
>> arm).
>
> That sounds reasonable.

I've been playing with this today, and that *seems* to work fine. At
least I can still build vmlinux, vmlinux.gz and it fixes the  build of
Image.gz as well..

I would appreciate some feedback on this one. I can then send a proper
patch. Could that break badly for anyone?

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 7a134d5c29..459b6d66a3 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -44,7 +44,10 @@ python __anonymous () {
         types = (alttype + ' ' + types).strip()
     d.setVar('KERNEL_IMAGETYPES', types)

-    typeformake = re.sub(r'\.gz', '', types)
+    # some commonly used kernel images aren't generated by the kernel
build system, such as vmlinux.gz
+    # typeformake lists only valid kernel make targets, and post
processing can be done after the kernel
+    # is built (such as using gzip to compress vmlinux)
+    typeformake = re.sub(r'vmlinux\.gz', 'vmlinux', types)
     d.setVar('KERNEL_IMAGETYPE_FOR_MAKE', typeformake)

     for type in types.split():


>
>> then a second step might be to check if we even need that mechanism
>> anymore.. but that might be a more tricky change.