[OE-core] Multilib status

Mark Hatle mark.hatle at windriver.com
Tue Sep 20 17:05:33 UTC 2011 

commit id aea5b9ef458099efd8f9a83428af84bbbe69eed2

I'm really not sure what I have is right though.. it seems to be not failing..
;)  But you likely know what should trigger these changes and if what I did is
correct or not.

--Mark

On 9/20/11 11:57 AM, Xu, Dongxiao wrote:
>>>> -----Original Message-----
>>>> From: Mark Hatle [mailto:mark.hatle at windriver.com]
>>>> Sent: Saturday, September 17, 2011 2:19 AM
>>>> To: Mark Hatle
>>>> Cc: Xu, Dongxiao; Richard Purdie; openembedded-core
>>>> Subject: Re: Multilib status
>>>>
>>>> Just an FYI.  I'm working through a bunch of issues.  I found out
>>>> that the RPM dependency resolution was completely broken, due to a
>>>> small bug in the package_rpm.bbclass..  Packages were not being given
>>>> the proper provide information, so RPM was attempting best match --
>>>> and failing as noted in the previous discussion..
>>>>
>>>> Unfortunately this has opened up a small can of worms which I'm
>>>> trying to deal with.  The simplistic fix for the bug is:
>>>>
>>>> --- a/meta/classes/package_rpm.bbclass
>>>> +++ b/meta/classes/package_rpm.bbclass
>>>> @@ -840,7 +840,7 @@ python do_package_rpm () {
>>>>         os.chmod(outdepends, 0755)
>>>>
>>>>         # Poky / RPM Provides
>>>> -       outprovides = workdir + "/" + srcname + ".requires"
>>>> +       outprovides = workdir + "/" + srcname + ".provides"
>>>>
>>>>         try:
>>>>                 from __builtin__ import file
>>>>
>>>> Unfortunately this has exposed a large set of problems that I didn't
>>>> realize we had....
>>>>
>>>> Will do a more formal pull request as soon as I get things working again.
>>>>
>>>> --Mark
>>>
>>> I just had a test after cherry-pick some of your changes in mhatle/rpm, here
>> are some problems I found and some investigations:
>>>
>>> 1. do_rootfs failed.
>>> The error log reports unmet dependency of pkgconfig(libpng).
>>> This is due to the dangling link in libpng package. libpng.pc links to libpng12.pc
>> which has been packaged into libpng12.
>>
>> Ahh this is a bit different then the failures I've seen.  We'll have to fix that one.
>>
>>> 2. There is a wrong commit I suppose it should be reverted. When you
>> debugging your code, please revert it.
>>> I will discuss it with Richard tomorrow.
>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=mhatl
>>> e/rpm&id=329d864f9bbf94ad3aae8df43d63fe10e4237e4f
>>
>> Ahh, ya I just ran into that problem... I have a fix for that commit I just finished.
>> (Do you have a newer version, if so I'll update to that and see if my change is still
>> needed.)
> 
> No I haven't worked on a new version of that patch. Could you share yours?
> 
> Thanks,
> Dongxiao
> 
>>
>>> 3. After the above issue got fixed, rootfs succeed but we saw the final image is
>> still in a mess. We need some x86_64 rpm to be installed, however what we got
>> is x86 rpm package. Then I tried another approach we ever talked about that,
>> firstly install base image, then install the multilib packages. I added a parameter
>> "--replacepkgs" to rpm command to avoid errors like "package xxx has already
>> installed". With this approach, the final image could boot up with multilib library
>> installed.
>>>
>>> Some of my testing code is located under
>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=dxu4/ml-
>>> testing
>>
>> I'll run through the testing code and see if I can figure out what is different.
>>
>> --Mark
>>
>>> Thanks,
>>> Dongxiao
>>>
>>>
>>>>
>>>> On 9/16/11 10:26 AM, Mark Hatle wrote:
>>>>> On 9/16/11 10:21 AM, Xu, Dongxiao wrote:
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Mark Hatle [mailto:mark.hatle at windriver.com]
>>>>>>>>> Sent: Friday, September 16, 2011 10:51 PM
>>>>>>>>> To: Richard Purdie
>>>>>>>>> Cc: Xu, Dongxiao; openembedded-core
>>>>>>>>> Subject: Re: Multilib status
>>>>>>>>>
>>>>>>>>> On 9/16/11 4:32 AM, Richard Purdie wrote:
>>>>>>>>>>> Hi Mark/Dongxaio,
>>>>>>>>>>>
>>>>>>>>>>> We really need to get the remainder of the multilib bugs
>>>>>>>>>>> wrapped up. I think there is some confusion about the exact
>>>>>>>>>>> approach to take to fix some of them so I'm hoping to try and
>>>>>>>>>>> summarise the problems here so we can work out some resolutions.
>>>>>>>>>
>>>>>>>>> Let me explain where I am quickly.  I just spent two days
>>>>>>>>> working on reproducing the issue(s).  So far I've not reproduced
>>>>>>>>> the direct failures but a series of others.  I finally figured
>>>>>>>>> out part of the reason I wasn't seeing this.  I had a typo in my
>>>> configuration:
>>>>>>>>>
>>>>>>>>> DEFAULTTUNE-virtclass-mulitlib-lib32 = "x86"
>>>>>>>>>
>>>>>>>>> This resulted in two sets of binaries normal and lib32 that were
>>>>>>>>> more or less identical in the RPM case.  We -really- need a
>>>>>>>>> sanity check that stupid typos like that don't cause problems.
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> So now that I finally have a built with that done... see below.
>>>>>>>>>
>>>>>>>>>>> Problem A
>>>>>>>>>>> =========
>>>>>>>>>>>
>>>>>>>>>>> We can have multiple forms of "machine specific" packages now,
>>>>>>>>>>> one for each multilib e.g.:
>>>>>>>>>>>
>>>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm ("normal")
>>>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm ("lib64")
>>>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm ("lib32")
>>>>>>>>>>>
>>>>>>>>>>> (lets assume the first uses /lib/, the second /lib64/ and the
>>>>>>>>>>> thrid /lib32/, an artificial example I realise)
>>>>>>>>>>>
>>>>>>>>>>> I know one proposal was to map:
>>>>>>>>>>>
>>>>>>>>>>> task-core-x11-base-1.0-r34.lib32_qemux86_64.rpm to
>>>>>>>>>>> task-core-x11-base-1.0-r34.lib32_qemux86.rpm but this isn't
>>>>>>>>>>> correct since the library directories are different. In this
>>>>>>>>>>> specific case it might not matter but in general it would.
>>>>>>>>>>> What is also important is that the different versions imply
>>>>>>>>>>> different dependencies. The lib64 bit version would pull in
>>>>>>>>>>> and select lib64 bit libs. Its these dependencies which are a
>>>>>>>>>>> key reason
>>>> these are not "all" arch packages.
>>>>>>>>>>>
>>>>>>>>>>> The end result is therefore we likely want:
>>>>>>>>>>>
>>>>>>>>>>> task-core-x11-base-1.0-r34.qemux86_64.rpm
>>>>>>>>>>> task-core-x11-base-1.0-r34.lib64_qemux86_64.rpm
>>>>>>>>>>> task-core-x11-base-1.0-r34.lib32_qemux86_64.rpm
>>>>>>>>>>>
>>>>>>>>>>> and I believe there are some patches Dongxiao has created
>>>>>>>>>>> which do
>>>> this.
>>>>>>>>>
>>>>>>>>> Yes, I'm currently testing his new MACHINE_virtclass-multilib-...
>>>>>>>>> patch.  So far it seems to be working.. assuming this is the
>>>>>>>>> approach we go with, we'll want to add some type of sanity check
>>>>>>>>> so that it's not possible to collide between the different
>>>>>>>>> multilib configurations.  (Alternatively we could, for each
>>>>>>>>> multilib,
>>>> specify a default machine virtclass in the format you list above.
>>>>>>>>>
>>>>>>>>> An alternative I was thinking of over night would be to avoid
>>>>>>>>> the RPM remapping on the "machine" packages.. but I'm not sure
>>>>>>>>> if that is really a good idea.
>>>>>>>>> However, it would could simplify the configuration aspects.
>>>>>>>
>>>>>>> There are two implementations towards the above result:
>>>>>>> 1) automatically setting MLPREFIX before MACHINE_ARCH.
>>>>>>> 2) user manually sets
>>>> MACHINE_virtclass-multilib-lib32="lib32-qemux86-64" in local.conf.
>>>>>>>
>>>>>>> Which one do you prefer?
>>>>> Right now I'm thinking the best approach is:
>>>>>
>>>>> *) Allow the user to set MACHINE_virtclass-multilib-...="...."
>>>>>
>>>>> *) If the user has NOT set it, fall back and use the setting of
>>>>> lib..-machine as in 2
>>>>>
>>>>> This way for people who have specific requirements for external
>>>>> compatibility or just desire more control they can do it.  But for
>>>>> everyone else it will still work properly.
>>>>>
>>>>> FYI, I'm currently working in the rootfs_rpm stuff, the system is
>>>>> currently ignoring the alternative (multilib) machine packages.  I'm
>>>>> hoping this will be fairly simple to resolve.
>>>>>
>>>>>>>>>
>>>>>>>>>>> Problem B
>>>>>>>>>>> =========
>>>>>>>>>>>
>>>>>>>>>>> If we install task-core-x11-base-1.0-r34.qemux86_64.rpm which
>>>>>>>>>>> depends on "connman", how does the system figure out to
>>>>>>>>>>> associate this to mean we want the "x86_64" architecture
>>>>>>>>>>> packages
>>>> installed?
>>>>>>>>>>>
>>>>>>>>>>> As far as I can tell there are two proposals around:
>>>>>>>>>>>
>>>>>>>>>>> 1) Set the architecture in the package provides and depends (a
>>>>>>>>>>> bit
>>>>>>>>>>> ugly)
>>>>>>>>>>>
>>>>>>>>>>> 2) Configure libzypp to understand that qemux86_64 does really
>>>>>>>>>>> map to
>>>>>>>>>>> x86_64 architecture and imply x86_64 dependencies.
>>>>>>>>>>>
>>>>>>>>>>> Solving problem A above is the first step towards resolving
>>>>>>>>>>> this either way but we don't seem to have a resolution on this
>>>>>>>>>>> second
>>>> part.
>>>>>>>>>
>>>>>>>>> Agreed.
>>>>>>>>>
>>>>>>>>>>> For reference, we really do want these task packages to have
>>>>>>>>>>> an associated architecture so the user can easily build up
>>>>>>>>>>> blocks of the system using a given ABI.
>>>>>>>>>
>>>>>>>>> This is going to be somewhat of a problem I suspect, simply
>>>>>>>>> because that is not the way RPM is/was designed.  We can adjust
>>>>>>>>> the rootfs install time, but this won't address field upgrade/installs.
>>>>>>>>>
>>>>>>>>> RPM (and Red Hat Linux/Fedora systems) appear to be designed
>>>>>>>>> that any package that reasonably meets the run-time dependencies
>>>>>>>>> can be
>>>> used.
>>>>>>>>> There is a concept of a "best" machine based on the resolver
>>>>>>>>> hierarchy, but ELF size may or may not be a factor in that decision.
>>>>>>>>>
>>>>>>>>> Doing this is quite powerful, but it also has a conscious
>>>>>>>>> decision set behind it.  If "bash" is required by a script, it
>>>>>>>>> really doesn't matter if it's ELF32,
>>>>>>>>> ELF64 or something else, as long as something provides bash.
>>>>>>>
>>>>>>> If user sets MULTILIB_IMAGE_INSTALL="lib32-task-core-boot" with a
>>>> qemux86-64 image, it means that he wants to install all elements
>>>> required by task-core-boot by lib32 version. If the dependency is
>>>> selected *randomly*, users would not have multiple libraries in his system.
>>>>> This is a special use case, yes one that I think we need to resolve,
>>>>> but it's not the typical case.
>>>>>
>>>>> The typical case will be someone wants the 64-bit (or 32-bit)
>>>>> version of a select package for compatibility.  And they only want
>>>>> the dependencies for that package to be loaded.  Anything satisfying
>>>>> the
>>>> dependency will do.
>>>>>
>>>>> My recommendation right now is, lets get that working ASAP.  Once
>>>>> that does, refocus the efforts on getting the multilib "tasks" to do
>>>>> something
>>>> reasonable.
>>>>> (i.e. select a group of packages)
>>>>>
>>>>> If we try to solve both at once, I don't think we're going to come
>>>>> up with a reasonable solution at this time.
>>>>>
>>>>> --Mark
>>>>>
>>>>>>> Thanks,
>>>>>>> Dongxiao
>>>>>>>>>
>>>>>>>>>>> What I really need now is an idea of which patches you both
>>>>>>>>>>> agree on that we need to merge (I think we have some for
>>>>>>>>>>> problem A) and a resolution on problem B along with some
>>>>>>>>>>> patches so we can close this set of issues out.
>>>>>>>>>>>
>>>>>>>>>>> If there are further issues can you reply to this email either
>>>>>>>>>>> with the details or a pointer to the specific additional problems.
>>>>>>>>>
>>>>>>>>> Now that I have my stupid typo out of the way, I expect to have
>>>>>>>>> further information in a few hours as to the regularly
>>>>>>>>> dependency resolution failure Dongxiao was reporting.  (Library
>>>>>>>>> dependencies
>>>>>>>>> -are- ELF specific, so those have to work properly!)
>>>>>>>>>
>>>>>>>>> --Mark
>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Richard
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>
>

More information about the Openembedded-core mailing list