[OE-core] [RFC][PATCH 2/2] buildhistory: support generating md5sum of files

[Jacob Kroon]

On Wed, Jan 9, 2019 at 12:20 PM Peter Kjellerstedt
<peter.kjellerstedt at axis.com> wrote:
>
> > -----Original Message-----
> > From: openembedded-core-bounces at lists.openembedded.org <openembedded-
> > core-bounces at lists.openembedded.org> On Behalf Of Jacob Kroon
> > Sent: den 6 januari 2019 19:14
> > To: openembedded-core at lists.openembedded.org
> > Subject: [OE-core] [RFC][PATCH 2/2] buildhistory: support generating
> > md5sum of files
> >
> > Introduce 'md5' in BUILDHISTORY_FEATURES and enable it by default
> > when doing reproducible builds.
> >
> > When enabled this will additionally create:
> >
> >   files-in-package-md5.txt
> >   files-in-image-md5.txt
> >   files-in-sdk-md5.txt
> >
> > containing the md5 checksums of regular files.
> >
> > Signed-off-by: Jacob Kroon <jacob.kroon at gmail.com>
> > ---
> >  meta/classes/buildhistory.bbclass | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes/buildhistory.bbclass
> > b/meta/classes/buildhistory.bbclass
> > index 33eb1b00f6..00f0701dec 100644
> > --- a/meta/classes/buildhistory.bbclass
> > +++ b/meta/classes/buildhistory.bbclass
> > @@ -7,7 +7,8 @@
> >  # Copyright (C) 2007-2011 Koen Kooi <koen at openembedded.org>
> >  #
> >
> > -BUILDHISTORY_FEATURES ?= "image package sdk"
> > +BUILDHISTORY_FEATURES ?= "image package sdk \
> > +  ${@ "md5" if bb.utils.to_boolean(d.getVar('BUILD_REPRODUCIBLE_BINARIES')) else ""}"
>
> May I suggest using ' instead of " within the Python block to
> lessen confusion when quotes are used within another set of quotes.
> (Even if bitbake handles it fine, not all editor coloring modes do.)
>

Ok.

> >  BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
> >  BUILDHISTORY_DIR_IMAGE = "${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/${IMAGE_BASENAME}"
> >  BUILDHISTORY_DIR_PACKAGE = "${BUILDHISTORY_DIR}/packages/${MULTIMACH_TARGET_SYS}/${PN}"
> > @@ -526,7 +527,12 @@ buildhistory_list_files() {
> >               eval ${FAKEROOTENV} ${FAKEROOTCMD} $find_cmd
> >       else
> >               eval $find_cmd
> > -     fi | sort -k5 | sed 's/ * -> $//' > $2 )
> > +     fi | sort -k5 | sed 's/ * -> $//' > $2
> > +     if [ "${@bb.utils.contains('BUILDHISTORY_FEATURES', 'md5', '1', '0', d)}" = "1" ] ; then
> > +             md5filename=$(echo $2 | sed 's/\.txt$/-md5.txt/')
> > +             find -type f | xargs -I{} -n1 md5sum {} | sort -k2 > $md5filename
> > +             [ -s $md5filename ] || rm $md5filename # remove result if empty
> > +     fi )
> >  }
> >
> >  buildhistory_list_pkg_files() {
> > --
> > 2.11.0
>
> Don't forget to update the documentation for BUILDHISTORY_FEATURES
> if this is added.
>

Oops, yeah I forgot that.

I'ill send a V2 with all comments addressed.

It does seem like once the equivalence work is ready and OE hashes its
outputs, much in buildhistory.bbclass can be replaced with already
prepared information. But for the cases where it is not used, and
considering how small the change is, maybe it makes sense to add it
anyway.

Thanks for the feedback,
Jacob

> //Peter
>