[OE-core] reproducible builds involving xz (was: Re: [PATCH v2] bitbake.conf: omit XZ threads and RAM from sstate signatures)

[André Draszik]

On Mon, 2020-02-24 at 17:10 +0200, Adrian Bunk wrote:
> On Mon, Feb 24, 2020 at 02:58:21PM +0000, André Draszik wrote:
> > On Mon, 2020-02-24 at 16:31 +0200, Adrian Bunk wrote:
> > > On Mon, Feb 24, 2020 at 02:21:57PM +0000, André Draszik wrote:
> > > > ...
> > > > Once the artefact has been created, it doesn't matter anymore how it
> > > > was created, as long as the high-level options aren't being changed (like
> > > > an explicit compression level, checksum method, etc.)
> > > > ...
> > > 
> > > meta-poky/conf/distro/poky.conf:INHERIT += "reproducible_build"
> > 
> > I still maintain these are two different problems. I am not trying to fix
> > reproducible builds.
> 
> But won't fixing reproducible builds require reverting the patch you 
> suggest for fixing your problem?

No, I don't think so.

To fix reproducible, you need to either patch xz to always behave like in
multi-threaded mode, even when --threads=1 was given in the arguments.
Additionally, a patch to xz to allow it to scale down the number of
threads, but not the compression level (--no-adjust prevents both),
could be useful.

Alternatively, you need to decide if you want to support it with
--threads=1 or --threads >= 2, and issue at least a warning in
reproducible_build.bbclass if somebody deviates.


You should also calculate the memory requirements, reduce the number
of threads to fit, and again give at least a warning if the compression
can not be done without changing compression level after that.

I still maintain that none of that is related to my original patch.



Cheers,
Andre'