[OE-core] [RFC PATCH] package.bbclass: improve -dbg and -src package ordering

[Joshua Watt]

This bug also affect sumo.

Armin, would you be able to you backport it please?

Thanks.

On Wed, 2018-07-11 at 13:38 +0200, Rasmus Villemoes wrote:
> nativesdk-gpgme fails package_qa when setting
> PACKAGE_DEBUG_SPLIT_STYLE
> = "debug-with-srcpkg".
> 
> ERROR: nativesdk-gpgme-1.10.0-r0 do_package_qa: QA Issue: non debug
> package contains .debug directory: nativesdk-python3-gpg path
> /work/x86_64-nativesdk-oesdk-linux/nativesdk-gpgme/1.10.0-
> r0/packages-split/nativesdk-python3-gpg/usr/local/oecore-
> x86_64/sysroots/x86_64-oesdk-linux/usr/lib/python3.5/site-
> packages/gpg/.debug/_gpgme.cpython-35m-x86_64-linux-gnu.so [debug-
> files]
> 
> This turns out to be because the automatic moving of the -dbg package
> to
> the beginning of the package list is disabled in that case, so the
> python3-gpg packages that the recipe prepends to PACKAGES ends up
> before
> the -dbg package.
> 
> It's not clear why the "and not split_source_package" was added when
> debug-with-srcpkg was introduced. Presumably the intention was to
> prevent the -dbg package to end up before the -src package, which we
> of
> course need to. But at the same time, we still need -dbg packages to
> end
> up before all other packages.
> 
> Using list.insert(0, ...) also means that if there happens to more
> than
> one -dbg package, their relative ordering gets inverted in the new
> list.
> 
> This tries to fix these issues by sorting the packages by (priority,
> original position), where priority is 10 for -src, 30 for -dbg and 50
> for everything else. That guarantees that packages of the same "type"
> preserve their relative ordering, while also ensuring that -dbg
> always
> preceed other packages. This scheme is also quite extensible, and,
> should the need arise, one could even expose the priorities as a knob
> the recipe author could use to ensure specific orderings of packages
> instead of the somewhat fragile and coarse-grained method of "prepend
> or
> append, and ensure you do that in a proper order".
> 
> Probably the autodebug condition needs to stay, but I think the
> split_source_package condition in the preceding elif should be
> removed,
> so that that logic applies to all packages called -src, not just the
> one
> we might have created a few lines above.
> 
> Signed-off-by: Rasmus Villemoes <rasmus.villemoes at prevas.dk>
> ---
>  meta/classes/package.bbclass | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/meta/classes/package.bbclass
> b/meta/classes/package.bbclass
> index 8276046369..72b9c66c3b 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -1151,21 +1151,22 @@ python populate_packages () {
>  
>      # Sanity check PACKAGES for duplicates
>      # Sanity should be moved to sanity.bbclass once we have the
> infrastructure
> -    package_list = []
> +    package_dict = {}
>  
> -    for pkg in packages.split():
> -        if pkg in package_list:
> +    for i, pkg in enumerate(packages.split()):
> +        if pkg in package_dict:
>              msg = "%s is listed in PACKAGES multiple times, this
> leads to packaging errors." % pkg
>              package_qa_handle_error("packages-list", msg, d)
>          # If debug-with-srcpkg mode is enabled then the src package
> will have
>          # priority over dbg package when assigning the files.
>          # This allows src package to include source files and remove
> them from dbg.
>          elif split_source_package and pkg.endswith("-src"):
> -            package_list.insert(0, pkg)
> -        elif autodebug and pkg.endswith("-dbg") and not
> split_source_package:
> -            package_list.insert(0, pkg)
> +            package_dict[pkg] = (10, i)
> +        elif autodebug and pkg.endswith("-dbg"):
> +            package_dict[pkg] = (30, i)
>          else:
> -            package_list.append(pkg)
> +            package_dict[pkg] = (50, i)
> +    package_list = sorted(package_dict.keys(), key=package_dict.get)
>      d.setVar('PACKAGES', ' '.join(package_list))
>      pkgdest = d.getVar('PKGDEST')
>  
> -- 
> 2.16.4
> 
-- 
Joshua Watt <JPEWhacker at gmail.com>