[OE-core] [PATCH 1/1] python-xcbgen_1.12: improve reproducibility

Bystricky, Juro juro.bystricky at intel.com
Wed Apr 11 15:26:52 UTC 2018


This would solve the .pyc timestamps problem. However, from reproducibility point of view, it is a bit more desirable to use python-native, as - in principle - different host Python versions can produce different compiled code as well. 
So it is not the timestamps only. 

________________________________________
From: Burton, Ross [ross.burton at intel.com]
Sent: Tuesday, April 10, 2018 5:02 PM
To: Bystricky, Juro
Cc: OE-core; Purdie, Richard; Juro Bystricky; Martin Jansa
Subject: Re: [PATCH 1/1] python-xcbgen_1.12: improve reproducibility

On 6 April 2018 at 20:57, Juro Bystricky <juro.bystricky at intel.com> wrote:
> In order to remove timestamps from all .pyc files we need to recompile
> them with python3-native, as we cannot rely on the host python being
> able to do that. Both python-native and python3-native derive the timestamp
> from SOURCE_DATE_EPOCH if present.
> However, building python3-native can be computationally expensive, so
> we resort to python3-native only when building reproducible packages:
> (BUILD_REPRODUCIBLE_BINARIES = '1'), otherwise we use the host python3.

The logical extension of this is that every recipe that ships .py
needs to depend on python3-native so that it can recompile.

However, there's another solution...

.pyc files are basically a four byte magic number, a four byte
timestamp, and then the byte code.  It won't be that many lines of
Python in a do_package[postfunc] to search for __pycache__/*.pyc files
in the packages and replace the timestamp with SOURCE_DATE_EPOCH if
reproducible builds are enabled.

Ross



More information about the Openembedded-core mailing list