[OE-core] [PATCH] classes/populate_sdk_base: Implement xz compression options

[Mike Looijmans]

On 19-03-2020 12:37, Adrian Bunk wrote:
> On Wed, Mar 18, 2020 at 01:21:56PM +0100, Mike Looijmans wrote:
>> Building an SDK on a machine with 8GB RAM resulted in excessive swapping
>> due to the xz compressor using ~20GB of memory. This is because xz is
>> being called with "-T 0 -9".
>>
>> To allow tuning the compression versus memory usage, introduce a variable
>> named SDK_XZ_OPTIONS that defaults to a more sane default:
>> SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${XZ_COMPRESSION_LEVEL}"
>> Thus, inherit any XZ tuning already done, and allow users to specify
>> overrides for this task in their config by supplying SDK_XZ_OPTIONS.
>> Since XZ_COMPRESSION_LEVEL defaults to -9 this does not change the standard
>> behavior.
>> ...
>> +SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${XZ_COMPRESSION_LEVEL}"
> 
> A problem is that XZ_COMPRESSION_LEVEL compression would now be used for
> unrelated usecases, and lowering the compression for being able to boot
> on low-end targets (<= 64 MB RAM) shouldn't impact the SDK.

Clear. I wasn't aware that this was for the targets...

> 
>> ...
>> -       d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; tar ${SDKTAROPTS} -cf - . | xz -T 0 -9 > ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}')
>> +       d.setVar('SDK_ARCHIVE_CMD', 'cd ${SDK_OUTPUT}/${SDKPATH}; tar ${SDKTAROPTS} -cf - . | xz ${SDK_XZ_OPTIONS} > ${SDKDEPLOYDIR}/${TOOLCHAIN_OUTPUTNAME}.${SDK_ARCHIVE_TYPE}')
>> ...
> 
> Replacing "-T 0" with ${XZ_DEFAULTS} should solve your problem.
> This already takes both cpus and memory into account.

I'd still want to have control over that -9 though.

It's not worth it to spend 15 minutes compressing just a few megabytes 
better in a development environment.


-- 
Mike Looijmans