[OE-core] About multilib packages rpm pkg name

[Mark Hatle]

On 8/5/15 9:33 PM, Robert Yang wrote:
> 
> Hi Mark,
> 
> Thank you very much, this makes things more clear now, I've tried
> to change packagegroup-core-standalone-sdk-target's PACKAGE_ARCH
> to MACHINE_ARCH or TUNE_PKGARCH, it didn't work (the rdepends lib32
> are still not installed), I will try to use read_subpkgdata() to get
> the required pkgs and fix the problem.

During package creation we strip off the multilib prefix (since it's not
needed).  Then during dependency calculations things often favor a 'shortest
path' to the dependency which may not give you the 32-bit version you are expecting.

What I'd like to see is a patch to smart where you can say "Use this ARCH or
word size, unless explicitly selected".  That would help eliminate -some- of
these issues.

The other side of the issue then is for things that do have explicit word size
requirements (like gcc and some of the build tools) we need to fix the packages
to have additions RDEPENDS and RPROVIDES that dictate the word size they need
and the word size they provide.  libpam already does this, and it resolved the
issues there.

--Mark

> // Robert
> 
> On 08/05/2015 09:10 PM, Mark Hatle wrote:
>> On 8/5/15 1:45 AM, Robert Yang wrote:
>>>
>>> Hello,
>>>
>>> For the multilib package like lib32-bash, its rpm package name
>>> is bash-4.3.30-r0.lib32_x86.rpm, but for ipk and deb, its name is
>>> lib32-bash.ipk and lib32-bash.deb, there is a side effect for
>>> the naming of rpm, for example, if packagegroup-xx rdepends on bash,
>>> the lib32-packagegroup-xx should rdepend on lib32-bash, but
>>> the its rpm pkg knows nothing about lib32-bash, so it still rdepends
>>> on bash, this causes 32bit toolchain libs not installed when export
>>> multilib sdk, for example:
>>>
>>> Configured multilib:
>>> MACHINE ?= "qemux86-64"
>>> require conf/multilib.conf
>>> MULTILIBS = "multilib:lib32"
>>> DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
>>>
>>> $ bitbake core-image-minimal -cpopulate_sdk
>>>
>>> When PACKAGE_CLASSES = "package_rpm",  the lib32-libgcc,
>>> lib32-libgcc-dev or other lib32-xx packages are not installed,
>>> so that the 32bit toolchain doesn't work in SDK, this is because
>>> lib32-packagegroup-core-standalone-sdk-target can't pull in the
>>> lib32-xx correctly.
>>>
>>> When PACKAGE_CLASSES = "package_ipk", it works well.
>>>
>>> Does anyone why we don't use the name like lib32-bash.rpm, please ?
>>> Can we use lib32-bash, please ?
>>
>> Because for something like bash, there is absolutely no reason to choose a
>> 64-bit or 32-bit version over the other.  They have equivalent functionality for
>> the end user, and only one version can be installed at a time.
>>
>> This is the RPM behavior that is being copied from Red Hat and other places, and
>> what our customers are expecting.
>>
>> RPM supports multiple packages, with different arch fields, being installed at
>> the same time.  ipkg and deb have a limitation that the package name MUST be
>> unique.  So this is the first reason why we have different style names.
>>
>> The idea here being that you can build the .i586 packages once, and re-use them
>> on an i586 system and an x86-64 system.  (Note, this doesn't work in practice in
>> the Yocto Project due to sstate issues... but that is the underlying design from
>> a package management perspective.)
>>
>> Dependency processing is the second reason, see below for an explanation of the
>> difference and how it affects us.
>>
>> There are different types of dependencies within the system.
>>
>> * Package level dependency
>> This is of the format 'bash' or 'lib32-bash'.  The system requires a specific
>> package.  All three package formats support this.
>>
>> * Virtual dependency
>> This is similar, in that the format can look like a package, or be something
>> like 'virtual/foo'.  Again all three package formats support this.
>>
>> * Library SONAME
>> This is a correlation of the library SONAME into the dependency set, such as
>> 'libz.so(64)'.  This is pulled directly from any ELF binaries in the system
>> during processing and added as a direct dependency to other shared libraries.
>> It is significantly more exact then the package or virtual dependencies above.
>> HOWEVER, it only works in the RPM case.
>>
>> This is the second reason why deb and ipkg need a unique 'lib32' style name for
>> things.  They don't have a mechanism for a package to say "I need the 64-bit
>> version of libz.so" to run properly, while that is built into RPM itself.
>>
>> * Filename dependency
>> These are dependencies that start with '/'.  This is unique to RPM as well, and
>> allows RPM to express dependencies on '/bin/bash', vs deb/ipkg having
>> dependencies on 'bash'.  The difference is that it puts a hard marker in place
>> that if 'bash' doesn't provide '/bin/bash', it's not enough.
>>
>> This doesn't directly affect the naming, but occasionally has some indirect
>> adjustments on things.
>>
>>
>> So when building a system using RPM as the package type, the rootfs construction
>> may be slightly different because the package manager rules are different.
>>
>> RPM (smart) will look at the overall dependency tree and attempt to pick the
>> shortest path so that all of the dependencies are correct.  You can 'weight' the
>> dependency resolver to favor one style of package over the other.  This is a
>> place in smart where we likely need to change the behavior to make it a bit more
>> friendly for the Yocto Project.
>>
>> The dependency generator is already configured to weigh a package of the same
>> type (arch) as a bit higher then others.  The problem is that a 'noarch' (all)
>> package is of a different type then say i586.  So when an all package says "give
>> me bash".  There is no 'bash.all', so it looks at the list and finds 'bash' of
>> the highest priority, IF it had not previously been selected by something else.
>>   (How would it have previously been selected?  If the image had selected
>> 'lib32-bash' explicitly, or the more likely case, a lib32-* package that
>> requires bash had been selected by the image, and there were not explicit
>> requirements for 'bash' [main lib version]).
>>
>>
>> RPM itself has a setting that defines which ABI type becomes the default if you
>> attempt to choose multiple packages that have the same executables listed.  The
>> setting here is:
>>
>> # Set RPM_PREFER_ELF_ARCH to configure preferred ABI when using rpm packaging
>> # backend to generate a rootfs, choices are:
>> # 1: ELF32 wins
>> # 2: ELF64 wins
>> # 4: ELF64 N32 wins (for mips64 or mips64el only)
>>
>> Note, this doesn't prevent multiple packages of the same name, different arch
>> from being installed -- what this does is selects which one wins.  When not set,
>> I believe it's last-in wins.
>>
>>
>> This setting should also help weigh the smart dep generator, but I'm not sure it
>> does.  (Pretty easy to test though.. set the value, build a multilib rootfs, and
>> select something (all package) that requires 'bash' and see which version is
>> selected.)
>>
>> But for most things in the system this really doesn't matter at all.  For things
>> where a 32-bit or 64-bit does matter (executable and/or libraries that are not
>> implicitly linked elsewhere), we really do need to add dependencies into the
>> system so that the correct set of items are selected.
>>
>> PAM, Perl, Python, and the on-target toolchains are things that adding a few
>> runtime dependencies are needed.  Basically anything in the system that may have
>> runtime loadable libraries of a specific ABI, using something like dlopen.
>> (Remember SONAME dependencies are automatic as mentioned above.)
>>
>> These new runtime dependencies will not harm deb/ipkg resolution (and may help
>> it), but they give RPM the additional information it needs to select the
>> explicit set that is required.  For example PAM:
>>
>> RPROVIDES_${PN} += "${PN}-${libpam_suffix}"
>> RPROVIDES_${PN}-runtime += "${PN}-runtime-${libpam_suffix}"
>>
>> RDEPENDS_${PN}-runtime = "${PN}-${libpam_suffix} \
>>      ${MLPREFIX}pam-plugin-deny-${libpam_suffix} \
>>      ${MLPREFIX}pam-plugin-permit-${libpam_suffix} \
>>      ${MLPREFIX}pam-plugin-warn-${libpam_suffix} \
>>      ${MLPREFIX}pam-plugin-unix-${libpam_suffix} \
>>      "
>>
>> (other code later constructs the RPROVIDES for the plugins as well)
>>
>> The MLPREFIX will be filtered off when RPM constructs it's packages, as it
>> doesn't want/need them.  However the suffix won't be filtered.
>>
>> libpam_suffix = "suffix${@get_multilib_bit(d)}"
>>
>> This way the -runtime package ensures the right other packages (with their
>> specific PROVIDES sufffixes) are included in all cases.
>>
>> This is a unique case, but should be pretty easy to help fix.
>>
>> For the case where you want to specify 'lib32-packagegroup' and expect to ONLY
>> get 32-bit packages... While we can't ensure it with the RPM mechanism, we can
>> weigh things to more likely happens -if- the lib32-packagegroup is of the same
>> .arch as the packages it's installing, instead of being 'all' packages.
>>
>> Hopefully this explains a bit more how all of these items fit together, and the
>> choice made when RPM was implemented to follow more of an expected multilib
>> workflow.  (This won't affect you if you don't use multilibs in your design!)
>>
>> --Mark
>>
>>