[OE-core] [PATCH 0/1] Initial QA test for reproducible builds
Joshua Watt
jpewhacker at gmail.com
Mon May 20 16:57:18 UTC 2019
Implements an initial QA check for reproducible builds. This check is
sufficient for an initial implementation, and will catch a wide variety
of reproducible problems, but it does have the following problems:
1) It doesn't pass. Currently, about 800 packages fail to build
in a reproducible manner for core-image-minimal. I've found two
major sources of non-reproducibility so far:
a) The perl-module packages don't have a consistent
SOURCE_DATE_EPOCH which means when they are packaged the
timestamps on all the files are different. Thankfully, this
accounts for several hundred of the packages, so fixing this
should remove a lot of the failures
b) Debug package strings aren't consistent. It appears that in some
of the -dbg packages, the linker changes the order of the merged
.debug_strings section. This trickles down into the packages
that contain the executables because it changes the hash the
executable contains to ensure the debug symbols match up.
2) It's not easy to debug issues when there are reproducibility
problems. I had initially intended to run diffoscope on the
resulting files but this takes much longer than I think we are
willing to run on the autobuilder and also generates far too much
output to be really useful. I think a better long term route is to
have the test dump the list of non-reproducible packages and then
write a helper script that can consumer this list, allow the user to
select a package, then run diffoscope to examine it.
3) This test currently is incomplete and won't catch all classes of
reproducibility problems. At the least, I know that it won't
consistently catch the use of the __DATE__ macro in source code,
since that requires the builds to be done on two separate dates (on
the other hand, use of __TIME__ will be caught pretty reliably since
the builds are done serially). I suspect the correct solution to
this is to borrow from Debian and use something like faketime to
fake out the system time to some suitable future date when doing the
test build, but this will require some though to how it should be
implemented.
4) It currently only tests Debian packages and core-image-minimal. The
test case has support for building the other package formats and
other images at the same time, the idea being that the long step in
this test is building everything from scratch, and building multiple
package formats and images at the same time will be much faster
overall than having multiple tests that have to do from-scratch
builds (although, there might be a way to serialize multiple tests
and have them share the test build TMPDIR). Until at least 1 package
format and image are passing, I don't see a huge motivation to
enable more.
Joshua Watt (1):
oeqa: Add reproducible build selftest
meta/lib/oeqa/selftest/cases/reproducible.py | 159 +++++++++++++++++++
1 file changed, 159 insertions(+)
create mode 100644 meta/lib/oeqa/selftest/cases/reproducible.py
--
2.21.0
More information about the Openembedded-core
mailing list