[OE-core] [PATCH v2] python3{,-native}: update to 3.7.0
Alejandro Enedino Hernandez Samaniego
alejandro.enedino.hernandez-samaniego at xilinx.com
Tue Sep 11 18:56:19 UTC 2018
Hey Jens,
On 09/10/2018 11:58 PM, Jens Rehsack wrote:
>
>
>> Am 10.09.2018 um 23:33 schrieb Alejandro Enedino Hernandez Samaniego
>> <alejandro.enedino.hernandez-samaniego at xilinx.com
>> <mailto:alejandro.enedino.hernandez-samaniego at xilinx.com>>:
>>
>> Hey Jens,
>>
>>
>> As I explained before, when you create a manifest for python
>> (target), it uses the native build as base (it literally runs the
>> native python that was just built), it is assumed its the same
>> version as target and contains all the modules provided by upstream,
>> otherwise the missing modules cannot be checked for dependencies, and
>> the manifest becomes incoherent, so its not an option to have an
>> incomplete python native build.
>>
> In that case, uuid for target never gets deployed, but it is. And I
> didn't see any packaging issues for `python3` nor for `nativesdk-python3`
I don't see what that has to do with anything, fixing the native build
should not affect what gets deployed on target, thats exactly why we
have a manifest, so they user can decide what to install and what not to
install.
I know you didn't see any packaging issues, but that doesn't mean they
don't exist, just from the log I showed you, I can tell you that the
python3-crypt package is not created correctly, for example, if you do:
IMAGE_INSTALL_append = " python3-crypt"
Boot the image, run python3 and you try to import sha3, it will fail,
because the sha3*.so library won't be on the filesystem.
And thats because the sha3.*.so library was just introduced in this
upgrade, and our manifest isn't aware it exists, hence it'll end up on
python3-misc and you have just created an unnecessary dependency from
python3-crypt to python3-misc (and worse, a dependency which were not
even aware of, until we test it manually), which beats the whole purpose
of the manifest.
The do_package function is not gonna fail just because, so you won't see
errors, but the files will be packaged incorrectly, causing runtime
errors as a consequence, the create_manifest task tries to solve these
runtime issues before they happen.
Cheers,
Alejandro
>> Yes you probably need a patch to look at the correct directories for
>> the h files, as well as a dependency to make the h files available on
>> recipe-sysroot-native.
>>
> Please check the submission.
I did, its not checking inside recipe-sysroot-native.
>
> Jens
>>
>> Alejandro
>>
>>
>> On 09/10/2018 02:05 PM, Jens Rehsack wrote:
>>> Hey Alejandro,
>>>
>>> I fixed that for cross-compile only, since I would need add a patch
>>> and a dependency python3-native for one thing: calculate uuids.
>>> When you can explain to me why the python-native needs that, I'll
>>> change that from -target & nativesdk to all.
>>>
>>> From my point of view it's not a question of having every (unneeded)
>>> python module being built for the native python, which is used for
>>> cross-compiling python and some modules only.
>>>
>>> Cheers,
>>> Jens
>>>
>>>> Am 10.09.2018 um 22:17 schrieb Alejandro Enedino Hernandez
>>>> Samaniego <alejandro.enedino.hernandez-samaniego at xilinx.com
>>>> <mailto:alejandro.enedino.hernandez-samaniego at xilinx.com>>:
>>>>
>>>> Hey Jens,
>>>>
>>>> The compilation log for python3-native still shows that it didn't
>>>> build the uuid module
>>>>
>>>>
>>>> Python build finished successfully!
>>>> The necessary bits to build these optional modules were not found:
>>>> _uuid
>>>>
>>>> Please look at my previous reply to find how this can be solved
>>>> (its likely a missing DEPENDS).
>>>>
>>>>
>>>> Also, this patch is missing the new python3 manifest for this
>>>> release, there appears to be a few new modules that we need to
>>>> decide which package they belong to, this is the output of bitbake
>>>> python3 -c create_manifest:
>>>>
>>>>
>>>> | ERROR:
>>>> | The following files are repeated (contained in more than one
>>>> package),
>>>> | this is likely to happen when new files are introduced after an
>>>> upgrade,
>>>> | please check which package should get it,
>>>> | modify the manifest accordingly and re-run the create_manifest task:
>>>> | ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so
>>>> | ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so
>>>> | ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so
>>>> | ${libdir}/python${PYTHON_MAJMIN}/contextvars.py
>>>> | ${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc
>>>> | ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so
>>>>
>>>>
>>>> Cheers,
>>>>
>>>>
>>>> Alejandro
>>>>
>>>>
>>>> On 09/10/2018 09:38 AM, Burton, Ross wrote:
>>>>> One thing to be aware of is that I've been fixing up Python's PGO
>>>>> support and there's a slew of patches in master-next and more just
>>>>> posted that this needs to be rebased on top of. Good news is that my
>>>>> patches remove two of the patches we've been carrying!
>>>>>
>>>>> Ross
>>>>>
>>>>> On 10 September 2018 at 17:36, Jens Rehsack <sno at netbsd.org
>>>>> <mailto:sno at netbsd.org>> wrote:
>>>>>>
>>>>>> Am 10.09.2018 um 11:35 schrieb Alexander Kanavin
>>>>>> <alex.kanavin at gmail.com <mailto:alex.kanavin at gmail.com>>:
>>>>>>
>>>>>> Large parts of dnf and friends have been rewritten in c++. I have not
>>>>>> yet updated and reviewed that, that will happen in the next cycle.
>>>>>>
>>>>>>
>>>>>> If I can prepare something for you - drop me a note.
>>>>>> Otherwise - the perl-5.28 update ("." in @INC, regex buffer
>>>>>> overflow, ...)
>>>>>> is also
>>>>>> awaiting some progress (I can keep "myself" busy).
>>>>>>
>>>>>> There's already enough disruption to deal with (postinsts errors,
>>>>>> openssl 1.1, both caused by me :)
>>>>>>
>>>>>>
>>>>>> You know, corner, ash, ... things happen. But there is progress!
>>>>>> Great!
>>>>>>
>>>>>> Good that we got all the way to do_rootfs though with 3.7.
>>>>>>
>>>>>>
>>>>>> Yeah, but than came postinst (coreutils :P) :D
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> Alex
>>>>>>
>>>>>> 2018-09-10 0:38 GMT+02:00 Tim Orling <ticotimo at gmail.com
>>>>>> <mailto:ticotimo at gmail.com>>:
>>>>>>
>>>>>> I did not review the patches closely, but I did try to build
>>>>>> core-image-full-cmdline with the tip of poky and these patches
>>>>>> applied.
>>>>>> Everything was fine until do_rootfs... I've attached the log.
>>>>>>
>>>>>> Essentially, there are some bits of dnf and so on which are not
>>>>>> ready for
>>>>>> Python 3.7. We have dnf version 2.7.5, but the latest upstream
>>>>>> release is
>>>>>> 3.4.0 (with a 3.5.0 just 3 days ago). Not sure yet if that would have
>>>>>> helped.
>>>>>>
>>>>>> --
>>>>>> _______________________________________________
>>>>>> Openembedded-core mailing list
>>>>>> Openembedded-core at lists.openembedded.org
>>>>>> <mailto:Openembedded-core at lists.openembedded.org>
>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jens Rehsack - rehsack at gmail.com
>>>>>>
>>>>>>
>>>>>> --
>>>>>> _______________________________________________
>>>>>> Openembedded-core mailing list
>>>>>> Openembedded-core at lists.openembedded.org
>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>>
>>>>
>>>> --
>>>> _______________________________________________
>>>> Openembedded-core mailing list
>>>> Openembedded-core at lists.openembedded.org
>>>> <mailto:Openembedded-core at lists.openembedded.org>
>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>
>>> --
>>> Jens Rehsack - rehsack at gmail.com <mailto:rehsack at gmail.com>
>>>
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> <mailto:Openembedded-core at lists.openembedded.org>
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
> --
> Jens Rehsack - rehsack at gmail.com <mailto:rehsack at gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20180911/e9b28ead/attachment-0002.html>
More information about the Openembedded-core
mailing list