[OE-core] [RFC] How to deal allarch packages in dependency chain for multilib rootfs?

[Mark Hatle]

On 6/28/18 3:48 AM, Kang Kai wrote:
> Hi all,
> 
> When build 32 bits rootfs with 64 bits bsp, if an allarch/noarch package is
> installed to lib32 rootfs, it causes
> unexpected 64 bits packages which is required by the allarch package installed
> to lib32 rootfs.
> 
> Take ca-certificates as example. ca-certificates rdepends on openssl, so if
> ca-certificates is installed to image,
> 64 bits package openssl will be installed too no matter what the rootfs is. But
> only 32 bits openssl package should
> be installed to 32 bits rootfs.

With -RPM-, and using names that do not include 'lib32' or other multilib
prefixes.  RPM doesn't care 'who' provides openssl, as long as someone provides
openssl.  There is a series of weights added to the available providers of
openssl.  The package with the highest weight (or maybe lowest) is what ends up
being installed.

In the case where there are two multilibs providing openssl, the system will
usually do one of two things:

1) Look to see if an existing openssl is set to be installed (just use that version)

2) Look to see if the default architecture has the package and use that,
otherwise fall back to less priority architectures.  (Again, this is stock RPM
behavior.)

OE has modified the behaviors slightly, so I can't be 100% confident as to how
it works in OE in all cases.

> There are 2 ways to fixed the issue.
> 
> 1 expand allarch/noarch packages with multilib. So if add ca-certificates to
> image, lib32-ca-certificates will be
>    installed to 32bits rootfs. And then also the dependency lib32-openssl is
> installed. That is what we expected.

This really isn't the right way to do it, because then you end up with two
packages with identical content -- and you then need to ensure that the internal
dependencies all 'match'.

So you've added another layer where you need to ensure that the multilib
rdepends are 'specified' and managed properly.  [Yes this can be handled by OE
automaticallly], but it doubles the size of the package respository (for no-arch
packages), as well as any field upgrades BOTH have to be upgraded together or
you will get an error.

> 2 expand DEPENDS/RDEPENDS of allarch/noarch packages with a prefix 'noarch-'
> when multilib is enabled. So then
>    ca-certificates requires 'noarch-openssl'. And make both lib32-openssl and
> openssl provides 'noarch-openssl'.

-today- it should be depending on 'openssl', and all variants of openssl SHOULD
be calling themselves/providing 'openssl'.  (The other components of the
dependency system will ensure the right 32-bit/64-bit version is selected if
needed.)

And unless the image creator is explicit in their check they may end up with
BOTH the 32-bit and 64-bit versions...

>    When do_rootfs, there is only one rpm repo 'oe-rootfs-repo' now. We will
> create repos with different priorities

And this is creating problems.  This is why the priority system (order for
dependencies) was put into place originally.  So it would start at the highest
priority components and work it's way until it found the right set of matches.

With DNF and RPM(4), things have changed..  I'm not sure how the priorities are
defined in this system -- but really each arch type should be it's own
repository in DNF.. and the priorities for each should be ordered to match the
user's preferred order for package installation.  (The default behavior is fine
for most users, but a way to say -- no really I want 32-bit installed first.. is
needed for the multilib image cases..)

The reality though for most multilib users is they have one of two use-cases:

1) 64-bit kernel, 32-bit userspace (and possibly a 'few' 64-bit binaries)

2) 64-bit kernel, 64-bit userspace... They want a small number of libraries from
the multilib installed into their main system

It's the second case where we often get into problems, because the system is
really intended for that -- where you would say I expect BOTH openssl 32-bit and
64-bit installed, but many of our users are trying to do I want only the openssl
installed for the executables I need.  (reasonable request, just not exactly
what RPM was intended to be doing.)

>    according to different archs/subdirectories. For 32 bits image, make 32 bits
> rpm repo has  higher priority, so lib32-openssl
>    will be installed to 32 bits rootfs rather than 64bits.

Yes, this covers the use case #1 better then anything else we have at this point.

> I know these 2 ways are not perfect, but only possible ways I have in mind to
> solve the problem.
> 
> Any comment or suggestion is greatly appreciated. Thanks a lot.

So I can sum this up to:

Use case 1.. kernel and primary userspace library type don't match is
problematic, but not used by a lot of people (AFAIK).

Use case 2.. the embedded nature of trying to develop 'small' [in terms of
number of packages installed] filesystems is causing some odd behavior, but a
workaround of specifying the -exact- packages you want when this oddity arrives
is available.

--Mark

> -- 
> Regards,
> Neil | Kai Kang
>