[OE-core] [PATCH V2] distutils/setuptools-native, distutils3/setuptools3-native: do not try to fetch code during do_compile
Derek Straka
derek at asterius.io
Fri Jul 27 12:43:19 UTC 2018
python-daemon - https://patchwork.openembedded.org/patch/153034/
python-stevedore - https://patchwork.openembedded.org/patch/153032/
I think that's everything though my world python build is still going.
Thanks for pointing the failures out and let me know if there are others.
I'm guessing this will ripple through the python heavy layers; I found a
couple yesterday in meta-iot-cloud.
On Fri, Jul 27, 2018 at 7:41 AM, Derek Straka <derek at asterius.io> wrote:
> Yes. I was just waiting for my master build to wrap up.
>
> On Fri, Jul 27, 2018, 7:39 AM Martin Jansa <martin.jansa at gmail.com> wrote:
>
>> I like this change, some recipes in meta-oe seems to be broken now, is
>> someone already working on a fix?
>>
>> I'm seeing following 5 failures in my world builds:
>>
>> python-stevedore-1.28.0-r0 do_compile and python3-stevedore-1.28.0-r0
>> do_compile
>> pkg_resources.DistributionNotFound: The 'pbr>=2.0.0' distribution was
>> not found and is required by the application
>>
>> python-dateutil-2.7.3-r0 do_compile and python3-dateutil-2.7.3-r0
>> do_compile
>> pkg_resources.DistributionNotFound: The 'setuptools_scm' distribution
>> was not found and is required by the application
>>
>> python-daemon-2.1.2-r0 do_compile
>> pkg_resources.DistributionNotFound: The 'docutils' distribution was not
>> found and is required by the application
>>
>> I've found a fix for python-dateutils from Derek:
>> https://patchwork.openembedded.org/patch/153002/
>>
>> On Tue, Jul 17, 2018 at 3:27 PM Hongxu Jia <hongxu.jia at windriver.com>
>> wrote:
>>
>>> If a python recipe is using setuptools and the setup_requires argument,
>>> where
>>> setuptools will use easy_install to fetch the module if it isn't
>>> present.
>>>
>>> The build failed on a machine where a proxy was required, but succeeded
>>> on a
>>> machine which had direct access to the internet
>>>
>>> Add var-NO_FETCH_BUILD, and set it in distutils_do_compile which does not
>>> allow to fetch code from internet during do_compile.
>>>
>>> Example result:
>>> ...
>>> ERROR: Do not try to fetch `pytest-runner1' for building. Please add its
>>> native recipe to DEPENDS.
>>> Traceback (most recent call last):
>>> File "setup.py", line 56, in <module>
>>> ...
>>>
>>> The improvement is flexible for test_requirements argument (used at
>>> `setup.py test'), where use easy_install also.
>>>
>>> [YOCTO #12084]
>>> Signed-off-by: Hongxu Jia <hongxu.jia at windriver.com>
>>> ---
>>> meta/classes/distutils.bbclass | 1 +
>>> meta/classes/distutils3.bbclass | 1 +
>>> ...ionally-do-not-fetch-code-by-easy_install.patch | 34
>>> ++++++++++++++++++++++
>>> meta/recipes-devtools/python/python-setuptools.inc | 2 ++
>>> 4 files changed, 38 insertions(+)
>>> create mode 100644 meta/recipes-devtools/python/
>>> files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>>>
>>> diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.
>>> bbclass
>>> index 1930c35..c2f7403 100644
>>> --- a/meta/classes/distutils.bbclass
>>> +++ b/meta/classes/distutils.bbclass
>>> @@ -8,6 +8,7 @@ DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \
>>> --install-data=${D}/${datadir}"
>>>
>>> distutils_do_compile() {
>>> + NO_FETCH_BUILD=1 \
>>> STAGING_INCDIR=${STAGING_INCDIR} \
>>> STAGING_LIBDIR=${STAGING_LIBDIR} \
>>> ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
>>> setup.py build ${DISTUTILS_BUILD_ARGS} || \
>>> diff --git a/meta/classes/distutils3.bbclass b/meta/classes/distutils3.
>>> bbclass
>>> index d4b92a3..f8aca5e 100644
>>> --- a/meta/classes/distutils3.bbclass
>>> +++ b/meta/classes/distutils3.bbclass
>>> @@ -15,6 +15,7 @@ distutils3_do_configure() {
>>> }
>>>
>>> distutils3_do_compile() {
>>> + NO_FETCH_BUILD=1 \
>>> STAGING_INCDIR=${STAGING_INCDIR} \
>>> STAGING_LIBDIR=${STAGING_LIBDIR} \
>>> ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
>>> setup.py \
>>> diff --git a/meta/recipes-devtools/python/files/0001-
>>> conditionally-do-not-fetch-code-by-easy_install.patch
>>> b/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-
>>> code-by-easy_install.patch
>>> new file mode 100644
>>> index 0000000..4b17921
>>> --- /dev/null
>>> +++ b/meta/recipes-devtools/python/files/0001-
>>> conditionally-do-not-fetch-code-by-easy_install.patch
>>> @@ -0,0 +1,34 @@
>>> +From 019422d19c6401fa8f45b1311ecd31bb087c809d Mon Sep 17 00:00:00 2001
>>> +From: Hongxu Jia <hongxu.jia at windriver.com>
>>> +Date: Tue, 17 Jul 2018 10:13:38 +0800
>>> +Subject: [PATCH] conditionally do not fetch code by easy_install
>>> +
>>> +If var-NO_FETCH_BUILD is set, do not allow to fetch code from
>>> +internet by easy_install.
>>> +
>>> +Upstream-Status: Inappropriate [oe specific]
>>> +
>>> +Signed-off-by: Hongxu Jia <hongxu.jia at windriver.com>
>>> +---
>>> + setuptools/command/easy_install.py | 5 +++++
>>> + 1 file changed, 5 insertions(+)
>>> +
>>> +diff --git a/setuptools/command/easy_install.py
>>> b/setuptools/command/easy_install.py
>>> +index 85ee40f..7eb9d2a 100755
>>> +--- a/setuptools/command/easy_install.py
>>> ++++ b/setuptools/command/easy_install.py
>>> +@@ -637,6 +637,11 @@ class easy_install(Command):
>>> + os.path.exists(tmpdir) and rmtree(rmtree_safe(tmpdir))
>>> +
>>> + def easy_install(self, spec, deps=False):
>>> ++ if os.environ.get('NO_FETCH_BUILD', None):
>>> ++ log.error("ERROR: Do not try to fetch `%s' for building. "
>>> ++ "Please add its native recipe to DEPENDS." %
>>> spec)
>>> ++ return None
>>> ++
>>> + if not self.editable:
>>> + self.install_site_py()
>>> +
>>> +--
>>> +2.7.4
>>> +
>>> diff --git a/meta/recipes-devtools/python/python-setuptools.inc
>>> b/meta/recipes-devtools/python/python-setuptools.inc
>>> index 25be446..24b3ac4 100644
>>> --- a/meta/recipes-devtools/python/python-setuptools.inc
>>> +++ b/meta/recipes-devtools/python/python-setuptools.inc
>>> @@ -9,6 +9,8 @@ PYPI_PACKAGE_EXT = "zip"
>>>
>>> inherit pypi
>>>
>>> +SRC_URI_append_class-native = " file://0001-conditionally-do-
>>> not-fetch-code-by-easy_install.patch"
>>> +
>>> SRC_URI[md5sum] = "260630ae1a64bafa39dcc53404d63829"
>>> SRC_URI[sha256sum] = "012adb8e25fbfd64c652e99e7bab58
>>> 799a3aaf05d39ab38561f69190a909015f"
>>>
>>> --
>>> 2.7.4
>>>
>>> --
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core at lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20180727/1ee71456/attachment-0002.html>
More information about the Openembedded-core
mailing list