[OE-core] [RFC PATCH] package.bbclass: improve -dbg and -src package ordering

[Joshua Watt]

On Mon, Sep 24, 2018 at 2:42 PM Joshua Watt <jpewhacker at gmail.com> wrote:
>
> This bug also affect sumo.
>
> Armin, would you be able to you backport it please?

Ping on the backport?

>
> Thanks.
>
> On Wed, 2018-07-11 at 13:38 +0200, Rasmus Villemoes wrote:
> > nativesdk-gpgme fails package_qa when setting
> > PACKAGE_DEBUG_SPLIT_STYLE
> > = "debug-with-srcpkg".
> >
> > ERROR: nativesdk-gpgme-1.10.0-r0 do_package_qa: QA Issue: non debug
> > package contains .debug directory: nativesdk-python3-gpg path
> > /work/x86_64-nativesdk-oesdk-linux/nativesdk-gpgme/1.10.0-
> > r0/packages-split/nativesdk-python3-gpg/usr/local/oecore-
> > x86_64/sysroots/x86_64-oesdk-linux/usr/lib/python3.5/site-
> > packages/gpg/.debug/_gpgme.cpython-35m-x86_64-linux-gnu.so [debug-
> > files]
> >
> > This turns out to be because the automatic moving of the -dbg package
> > to
> > the beginning of the package list is disabled in that case, so the
> > python3-gpg packages that the recipe prepends to PACKAGES ends up
> > before
> > the -dbg package.
> >
> > It's not clear why the "and not split_source_package" was added when
> > debug-with-srcpkg was introduced. Presumably the intention was to
> > prevent the -dbg package to end up before the -src package, which we
> > of
> > course need to. But at the same time, we still need -dbg packages to
> > end
> > up before all other packages.
> >
> > Using list.insert(0, ...) also means that if there happens to more
> > than
> > one -dbg package, their relative ordering gets inverted in the new
> > list.
> >
> > This tries to fix these issues by sorting the packages by (priority,
> > original position), where priority is 10 for -src, 30 for -dbg and 50
> > for everything else. That guarantees that packages of the same "type"
> > preserve their relative ordering, while also ensuring that -dbg
> > always
> > preceed other packages. This scheme is also quite extensible, and,
> > should the need arise, one could even expose the priorities as a knob
> > the recipe author could use to ensure specific orderings of packages
> > instead of the somewhat fragile and coarse-grained method of "prepend
> > or
> > append, and ensure you do that in a proper order".
> >
> > Probably the autodebug condition needs to stay, but I think the
> > split_source_package condition in the preceding elif should be
> > removed,
> > so that that logic applies to all packages called -src, not just the
> > one
> > we might have created a few lines above.
> >
> > Signed-off-by: Rasmus Villemoes <rasmus.villemoes at prevas.dk>
> > ---
> >  meta/classes/package.bbclass | 15 ++++++++-------
> >  1 file changed, 8 insertions(+), 7 deletions(-)
> >
> > diff --git a/meta/classes/package.bbclass
> > b/meta/classes/package.bbclass
> > index 8276046369..72b9c66c3b 100644
> > --- a/meta/classes/package.bbclass
> > +++ b/meta/classes/package.bbclass
> > @@ -1151,21 +1151,22 @@ python populate_packages () {
> >
> >      # Sanity check PACKAGES for duplicates
> >      # Sanity should be moved to sanity.bbclass once we have the
> > infrastructure
> > -    package_list = []
> > +    package_dict = {}
> >
> > -    for pkg in packages.split():
> > -        if pkg in package_list:
> > +    for i, pkg in enumerate(packages.split()):
> > +        if pkg in package_dict:
> >              msg = "%s is listed in PACKAGES multiple times, this
> > leads to packaging errors." % pkg
> >              package_qa_handle_error("packages-list", msg, d)
> >          # If debug-with-srcpkg mode is enabled then the src package
> > will have
> >          # priority over dbg package when assigning the files.
> >          # This allows src package to include source files and remove
> > them from dbg.
> >          elif split_source_package and pkg.endswith("-src"):
> > -            package_list.insert(0, pkg)
> > -        elif autodebug and pkg.endswith("-dbg") and not
> > split_source_package:
> > -            package_list.insert(0, pkg)
> > +            package_dict[pkg] = (10, i)
> > +        elif autodebug and pkg.endswith("-dbg"):
> > +            package_dict[pkg] = (30, i)
> >          else:
> > -            package_list.append(pkg)
> > +            package_dict[pkg] = (50, i)
> > +    package_list = sorted(package_dict.keys(), key=package_dict.get)
> >      d.setVar('PACKAGES', ' '.join(package_list))
> >      pkgdest = d.getVar('PKGDEST')
> >
> > --
> > 2.16.4
> >
> --
> Joshua Watt <JPEWhacker at gmail.com>