[OE-core] Multilib status

Mark Hatle mark.hatle at windriver.com
Tue Sep 20 16:52:36 UTC 2011 

On 9/20/11 10:01 AM, Xu, Dongxiao wrote:
> Hi Mark,
> 
>> -----Original Message-----
>> From: Mark Hatle [mailto:mark.hatle at windriver.com]
>> Sent: Saturday, September 17, 2011 2:19 AM
>> To: Mark Hatle
>> Cc: Xu, Dongxiao; Richard Purdie; openembedded-core
>> Subject: Re: Multilib status
>>
>> Just an FYI.  I'm working through a bunch of issues.  I found out that the RPM
>> dependency resolution was completely broken, due to a small bug in the
>> package_rpm.bbclass..  Packages were not being given the proper provide
>> information, so RPM was attempting best match -- and failing as noted in the
>> previous discussion..
>>
>> Unfortunately this has opened up a small can of worms which I'm trying to deal
>> with.  The simplistic fix for the bug is:
>>
>> --- a/meta/classes/package_rpm.bbclass
>> +++ b/meta/classes/package_rpm.bbclass
>> @@ -840,7 +840,7 @@ python do_package_rpm () {
>>         os.chmod(outdepends, 0755)
>>
>>         # Poky / RPM Provides
>> -       outprovides = workdir + "/" + srcname + ".requires"
>> +       outprovides = workdir + "/" + srcname + ".provides"
>>
>>         try:
>>                 from __builtin__ import file
>>
>> Unfortunately this has exposed a large set of problems that I didn't realize we
>> had....
>>
>> Will do a more formal pull request as soon as I get things working again.
>>
>> --Mark
> 
> I just had a test after cherry-pick some of your changes in mhatle/rpm, here are some problems I found and some investigations:
> 
> 1. do_rootfs failed.
> The error log reports unmet dependency of pkgconfig(libpng).
> This is due to the dangling link in libpng package. libpng.pc links to libpng12.pc which has been packaged into libpng12.

Ahh this is a bit different then the failures I've seen.  We'll have to fix that
one.

> 2. There is a wrong commit I suppose it should be reverted. When you debugging your code, please revert it.
> I will discuss it with Richard tomorrow.
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=mhatle/rpm&id=329d864f9bbf94ad3aae8df43d63fe10e4237e4f

Ahh, ya I just ran into that problem... I have a fix for that commit I just
finished.  (Do you have a newer version, if so I'll update to that and see if my
change is still needed.)

> 3. After the above issue got fixed, rootfs succeed but we saw the final image is still in a mess. We need some x86_64 rpm to be installed, however what we got is x86 rpm package. Then I tried another approach we ever talked about that, firstly install base image, then install the multilib packages. I added a parameter "--replacepkgs" to rpm command to avoid errors like "package xxx has already installed". With this approach, the final image could boot up with multilib library installed.
> 
> Some of my testing code is located under http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=dxu4/ml-testing

I'll run through the testing code and see if I can figure out what is different.

--Mark

> Thanks,
> Dongxiao
> 
> 	
>>
>> On 9/16/11 10:26 AM, Mark Hatle wrote:
>>> On 9/16/11 10:21 AM, Xu, Dongxiao wrote:
>>>>>>> -----Original Message-----
>>>>>>> From: Mark Hatle [mailto:mark.hatle at windriver.com]
>>>>>>> Sent: Friday, September 16, 2011 10:51 PM
>>>>>>> To: Richard Purdie
>>>>>>> Cc: Xu, Dongxiao; openembedded-core
>>>>>>> Subject: Re: Multilib status
>>>>>>>
>>>>>>> On 9/16/11 4:32 AM, Richard Purdie wrote:
>>>>>>>>> Hi Mark/Dongxaio,
>>>>>>>>>
>>>>>>>>> We really need to get the remainder of the multilib bugs
>>>>>>>>> wrapped up. I think there is some confusion about the exact
>>>>>>>>> approach to take to fix some of them so I'm hoping to try and
>>>>>>>>> summarise the problems here so we can work out some resolutions.
>>>>>>>
>>>>>>> Let me explain where I am quickly.  I just spent two days working
>>>>>>> on reproducing the issue(s).  So far I've not reproduced the
>>>>>>> direct failures but a series of others.  I finally figured out
>>>>>>> part of the reason I wasn't seeing this.  I had a typo in my
>> configuration:
>>>>>>>
>>>>>>> DEFAULTTUNE-virtclass-mulitlib-lib32 = "x86"
>>>>>>>
>>>>>>> This resulted in two sets of binaries normal and lib32 that were
>>>>>>> more or less identical in the RPM case.  We -really- need a
>>>>>>> sanity check that stupid typos like that don't cause problems.
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> So now that I finally have a built with that done... see below.
>>>>>>>
>>>>>>>>> Problem A
>>>>>>>>> =========
>>>>>>>>>
>>>>>>>>> We can have multiple forms of "machine specific" packages now,
>>>>>>>>> one for each multilib e.g.:
>>>>>>>>>
>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm ("normal")
>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm ("lib64")
>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm ("lib32")
>>>>>>>>>
>>>>>>>>> (lets assume the first uses /lib/, the second /lib64/ and the
>>>>>>>>> thrid /lib32/, an artificial example I realise)
>>>>>>>>>
>>>>>>>>> I know one proposal was to map:
>>>>>>>>>
>>>>>>>>> task-core-x11-base-1.0-r34.lib32_qemux86_64.rpm to
>>>>>>>>> task-core-x11-base-1.0-r34.lib32_qemux86.rpm but this isn't
>>>>>>>>> correct since the library directories are different. In this
>>>>>>>>> specific case it might not matter but in general it would. What
>>>>>>>>> is also important is that the different versions imply
>>>>>>>>> different dependencies. The lib64 bit version would pull in and
>>>>>>>>> select lib64 bit libs. Its these dependencies which are a key reason
>> these are not "all" arch packages.
>>>>>>>>>
>>>>>>>>> The end result is therefore we likely want:
>>>>>>>>>
>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm
>>>>>>>>> task-core-x11-base-1.0-r34.lib64_qemux86_64.rpm
>>>>>>>>> task-core-x11-base-1.0-r34.lib32_qemux86_64.rpm
>>>>>>>>>
>>>>>>>>> and I believe there are some patches Dongxiao has created which do
>> this.
>>>>>>>
>>>>>>> Yes, I'm currently testing his new MACHINE_virtclass-multilib-...
>>>>>>> patch.  So far it seems to be working.. assuming this is the
>>>>>>> approach we go with, we'll want to add some type of sanity check
>>>>>>> so that it's not possible to collide between the different
>>>>>>> multilib configurations.  (Alternatively we could, for each multilib,
>> specify a default machine virtclass in the format you list above.
>>>>>>>
>>>>>>> An alternative I was thinking of over night would be to avoid the
>>>>>>> RPM remapping on the "machine" packages.. but I'm not sure if
>>>>>>> that is really a good idea.
>>>>>>> However, it would could simplify the configuration aspects.
>>>>>
>>>>> There are two implementations towards the above result:
>>>>> 1) automatically setting MLPREFIX before MACHINE_ARCH.
>>>>> 2) user manually sets
>> MACHINE_virtclass-multilib-lib32="lib32-qemux86-64" in local.conf.
>>>>>
>>>>> Which one do you prefer?
>>> Right now I'm thinking the best approach is:
>>>
>>> *) Allow the user to set MACHINE_virtclass-multilib-...="...."
>>>
>>> *) If the user has NOT set it, fall back and use the setting of
>>> lib..-machine as in 2
>>>
>>> This way for people who have specific requirements for external
>>> compatibility or just desire more control they can do it.  But for
>>> everyone else it will still work properly.
>>>
>>> FYI, I'm currently working in the rootfs_rpm stuff, the system is
>>> currently ignoring the alternative (multilib) machine packages.  I'm
>>> hoping this will be fairly simple to resolve.
>>>
>>>>>>>
>>>>>>>>> Problem B
>>>>>>>>> =========
>>>>>>>>>
>>>>>>>>> If we install task-core-x11-base-1.0-r34.qemux86_64.rpm which
>>>>>>>>> depends on "connman", how does the system figure out to
>>>>>>>>> associate this to mean we want the "x86_64" architecture packages
>> installed?
>>>>>>>>>
>>>>>>>>> As far as I can tell there are two proposals around:
>>>>>>>>>
>>>>>>>>> 1) Set the architecture in the package provides and depends (a
>>>>>>>>> bit
>>>>>>>>> ugly)
>>>>>>>>>
>>>>>>>>> 2) Configure libzypp to understand that qemux86_64 does really
>>>>>>>>> map to
>>>>>>>>> x86_64 architecture and imply x86_64 dependencies.
>>>>>>>>>
>>>>>>>>> Solving problem A above is the first step towards resolving
>>>>>>>>> this either way but we don't seem to have a resolution on this second
>> part.
>>>>>>>
>>>>>>> Agreed.
>>>>>>>
>>>>>>>>> For reference, we really do want these task packages to have an
>>>>>>>>> associated architecture so the user can easily build up blocks
>>>>>>>>> of the system using a given ABI.
>>>>>>>
>>>>>>> This is going to be somewhat of a problem I suspect, simply
>>>>>>> because that is not the way RPM is/was designed.  We can adjust
>>>>>>> the rootfs install time, but this won't address field upgrade/installs.
>>>>>>>
>>>>>>> RPM (and Red Hat Linux/Fedora systems) appear to be designed that
>>>>>>> any package that reasonably meets the run-time dependencies can be
>> used.
>>>>>>> There is a concept of a "best" machine based on the resolver
>>>>>>> hierarchy, but ELF size may or may not be a factor in that decision.
>>>>>>>
>>>>>>> Doing this is quite powerful, but it also has a conscious
>>>>>>> decision set behind it.  If "bash" is required by a script, it
>>>>>>> really doesn't matter if it's ELF32,
>>>>>>> ELF64 or something else, as long as something provides bash.
>>>>>
>>>>> If user sets MULTILIB_IMAGE_INSTALL="lib32-task-core-boot" with a
>> qemux86-64 image, it means that he wants to install all elements required by
>> task-core-boot by lib32 version. If the dependency is selected *randomly*, users
>> would not have multiple libraries in his system.
>>> This is a special use case, yes one that I think we need to resolve,
>>> but it's not the typical case.
>>>
>>> The typical case will be someone wants the 64-bit (or 32-bit) version
>>> of a select package for compatibility.  And they only want the
>>> dependencies for that package to be loaded.  Anything satisfying the
>> dependency will do.
>>>
>>> My recommendation right now is, lets get that working ASAP.  Once that
>>> does, refocus the efforts on getting the multilib "tasks" to do something
>> reasonable.
>>> (i.e. select a group of packages)
>>>
>>> If we try to solve both at once, I don't think we're going to come up
>>> with a reasonable solution at this time.
>>>
>>> --Mark
>>>
>>>>> Thanks,
>>>>> Dongxiao
>>>>>>>
>>>>>>>>> What I really need now is an idea of which patches you both
>>>>>>>>> agree on that we need to merge (I think we have some for
>>>>>>>>> problem A) and a resolution on problem B along with some
>>>>>>>>> patches so we can close this set of issues out.
>>>>>>>>>
>>>>>>>>> If there are further issues can you reply to this email either
>>>>>>>>> with the details or a pointer to the specific additional problems.
>>>>>>>
>>>>>>> Now that I have my stupid typo out of the way, I expect to have
>>>>>>> further information in a few hours as to the regularly dependency
>>>>>>> resolution failure Dongxiao was reporting.  (Library dependencies
>>>>>>> -are- ELF specific, so those have to work properly!)
>>>>>>>
>>>>>>> --Mark
>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Richard
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>

More information about the Openembedded-core mailing list