[bitbake-devel] can i get a list of only bbappend'ed recipes that were *used* in a build?

[Peter Kjellerstedt]

> -----Original Message-----
> From: bitbake-devel-bounces at lists.openembedded.org <bitbake-devel-
> bounces at lists.openembedded.org> On Behalf Of Robert P. J. Day
> Sent: den 28 februari 2020 20:38
> To: Mikko.Rapeli at bmw.de
> Cc: bitbake-devel at lists.openembedded.org
> Subject: Re: [bitbake-devel] can i get a list of only bbappend'ed recipes
> that were *used* in a build?
> 
> On Fri, 28 Feb 2020, Mikko.Rapeli at bmw.de wrote:
> 
> > Hi,
> >
> > On Fri, Feb 28, 2020 at 08:33:36AM -0500, rpjday at crashcourse.ca wrote:
> > > looked around and i don't see an immediately obvious way to do
> > > this ... is there a way to generate a list of *only* those recipes
> > > that were used in the current build that had associated .bbappend
> > > files?
> > >
> > > "bitbake-layers show-appends" will show all recipes with .bbappend
> > > files, including those that are not included and processed for the
> > > current build/image. here's the rationale, in case i'm missing
> > > something.
> >
> > I'm doing "find meta* -name \*recipe\*bb\*" and reviewing every one
> > of them. More visibility to this including parsing order would be
> > nice to store in e.g. buildhistory.
> 
>   thanks for clarifying that there wasn't a massively trivial way to
> do this that i just didn't see.

For each image you build, you should find a manifest in 
"tmp/deploy/image/<machine>/<image>-<machine>.manifest". It contains the 
names, architectures and versions of all packages that were installed in 
the image. 

To convert the package names from the manifest to recipes, you can use 
oe-pkgdata-util:

  oe-pkgdata-util -p tmp/pkgdata/<machine> package-info -f <path to the manifest file>

(the -p tmp/pkgdata/<machine> option isn't necessary, but you avoid 
starting tinfoil if you use it). This will give you one line per package 
that also includes the recipe name. Pass the output to `cut -f3 -d' ' | 
sort -u' and you should have the list of recipes that were used to build 
the packages in your image.

Then all you have to do is take the intersection of recipes from that 
list and from `bitbake-layers show-appends` and you should have the 
list of recipes that have bbappends applied and are used to build your 
image.

Not trivial, but it should be possible to write a relatively simple 
script to do the above.

> > > been handed an existing vendor layer, with *tons* of bbappend'ed
> > > recipes, some basic, some from meta-openembedded and many from the
> > > vendor's proprietary layer. my go-to strategy is to always try to
> > > build a core-image-minimal to establish a working baseline and,
> > > when that works, it would be nice to know which of the recipes
> > > that contributed to that core-image-minimal build were modified in
> > > any way via a .bbappend file, then work my way out.
> > >
> > > does that make sense? is there a way to do this, other than wander
> > > down to tmp/work/... and try to figure out which recipes were
> > > modified from the original? am i overlooking something simple?
> >
> > I start off by reviewing every single file in vendor and BSP SW
> > layers before I add them to builds. Sadly this does not catch bad
> > things which get integrated later in updates.
> >
> > Resulting distro config in my projects is full of BBMASK paths to
> > the various layers which are intrusive with their bbappends.
> 
> yup, i'd already thought of that ... a methodical way to slowly add
> recipes/layers to the build.
> 
> > Maybe recording all bbappends and their relative paths in the
> > project and parsing order to buildhistory would help.
> >
> > I would also like to limit layers from adding bbappends to certain
> > recipes but I don't see ways for this currently without external
> > scripts.
> 
> yeah, there's lots of cool things i can think of that would assist
> in a migration from an older version of OE/YP to a more recent one.
> 
> rday

//Peter