[OE-core] [PATCH V2] distutils/setuptools-native, distutils3/setuptools3-native: do not try to fetch code during do_compile

Fri Jul 27 11:41:35 UTC 2018

Yes.  I was just waiting for my master build to wrap up.

On Fri, Jul 27, 2018, 7:39 AM Martin Jansa <martin.jansa at gmail.com> wrote:

> I like this change, some recipes in meta-oe seems to be broken now, is
> someone already working on a fix?
>
> I'm seeing following 5 failures in my world builds:
>
> python-stevedore-1.28.0-r0 do_compile and python3-stevedore-1.28.0-r0
> do_compile
> pkg_resources.DistributionNotFound: The 'pbr>=2.0.0' distribution was not
> found and is required by the application
>
> python-dateutil-2.7.3-r0 do_compile and python3-dateutil-2.7.3-r0
> do_compile
> pkg_resources.DistributionNotFound: The 'setuptools_scm' distribution was
> not found and is required by the application
>
> python-daemon-2.1.2-r0 do_compile
> pkg_resources.DistributionNotFound: The 'docutils' distribution was not
> found and is required by the application
>
> I've found a fix for python-dateutils from Derek:
> https://patchwork.openembedded.org/patch/153002/
>
> On Tue, Jul 17, 2018 at 3:27 PM Hongxu Jia <hongxu.jia at windriver.com>
> wrote:
>
>> If a python recipe is using setuptools and the setup_requires argument,
>> where
>> setuptools will use easy_install to fetch the module if it isn't
>> present.
>>
>> The build failed on a machine where a proxy was required, but succeeded
>> on a
>> machine which had direct access to the internet
>>
>> Add var-NO_FETCH_BUILD, and set it in distutils_do_compile which does not
>> allow to fetch code from internet during do_compile.
>>
>> Example result:
>> ...
>> ERROR: Do not try to fetch `pytest-runner1' for building. Please add its
>> native recipe to DEPENDS.
>> Traceback (most recent call last):
>>   File "setup.py", line 56, in <module>
>> ...
>>
>> The improvement is flexible for test_requirements argument (used at
>> `setup.py test'), where use easy_install also.
>>
>> [YOCTO #12084]
>> Signed-off-by: Hongxu Jia <hongxu.jia at windriver.com>
>> ---
>>  meta/classes/distutils.bbclass                     |  1 +
>>  meta/classes/distutils3.bbclass                    |  1 +
>>  ...ionally-do-not-fetch-code-by-easy_install.patch | 34
>> ++++++++++++++++++++++
>>  meta/recipes-devtools/python/python-setuptools.inc |  2 ++
>>  4 files changed, 38 insertions(+)
>>  create mode 100644
>> meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>>
>> diff --git a/meta/classes/distutils.bbclass
>> b/meta/classes/distutils.bbclass
>> index 1930c35..c2f7403 100644
>> --- a/meta/classes/distutils.bbclass
>> +++ b/meta/classes/distutils.bbclass
>> @@ -8,6 +8,7 @@ DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \
>>      --install-data=${D}/${datadir}"
>>
>>  distutils_do_compile() {
>> +         NO_FETCH_BUILD=1 \
>>           STAGING_INCDIR=${STAGING_INCDIR} \
>>           STAGING_LIBDIR=${STAGING_LIBDIR} \
>>           ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
>> setup.py build ${DISTUTILS_BUILD_ARGS} || \
>> diff --git a/meta/classes/distutils3.bbclass
>> b/meta/classes/distutils3.bbclass
>> index d4b92a3..f8aca5e 100644
>> --- a/meta/classes/distutils3.bbclass
>> +++ b/meta/classes/distutils3.bbclass
>> @@ -15,6 +15,7 @@ distutils3_do_configure() {
>>  }
>>
>>  distutils3_do_compile() {
>> +        NO_FETCH_BUILD=1 \
>>          STAGING_INCDIR=${STAGING_INCDIR} \
>>          STAGING_LIBDIR=${STAGING_LIBDIR} \
>>          ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}
>> setup.py \
>> diff --git
>> a/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>> b/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>> new file mode 100644
>> index 0000000..4b17921
>> --- /dev/null
>> +++
>> b/meta/recipes-devtools/python/files/0001-conditionally-do-not-fetch-code-by-easy_install.patch
>> @@ -0,0 +1,34 @@
>> +From 019422d19c6401fa8f45b1311ecd31bb087c809d Mon Sep 17 00:00:00 2001
>> +From: Hongxu Jia <hongxu.jia at windriver.com>
>> +Date: Tue, 17 Jul 2018 10:13:38 +0800
>> +Subject: [PATCH] conditionally do not fetch code by easy_install
>> +
>> +If var-NO_FETCH_BUILD is set, do not allow to fetch code from
>> +internet by easy_install.
>> +
>> +Upstream-Status: Inappropriate [oe specific]
>> +
>> +Signed-off-by: Hongxu Jia <hongxu.jia at windriver.com>
>> +---
>> + setuptools/command/easy_install.py | 5 +++++
>> + 1 file changed, 5 insertions(+)
>> +
>> +diff --git a/setuptools/command/easy_install.py
>> b/setuptools/command/easy_install.py
>> +index 85ee40f..7eb9d2a 100755
>> +--- a/setuptools/command/easy_install.py
>> ++++ b/setuptools/command/easy_install.py
>> +@@ -637,6 +637,11 @@ class easy_install(Command):
>> +             os.path.exists(tmpdir) and rmtree(rmtree_safe(tmpdir))
>> +
>> +     def easy_install(self, spec, deps=False):
>> ++        if os.environ.get('NO_FETCH_BUILD', None):
>> ++            log.error("ERROR: Do not try to fetch `%s' for building. "
>> ++                      "Please add its native recipe to DEPENDS." % spec)
>> ++            return None
>> ++
>> +         if not self.editable:
>> +             self.install_site_py()
>> +
>> +--
>> +2.7.4
>> +
>> diff --git a/meta/recipes-devtools/python/python-setuptools.inc
>> b/meta/recipes-devtools/python/python-setuptools.inc
>> index 25be446..24b3ac4 100644
>> --- a/meta/recipes-devtools/python/python-setuptools.inc
>> +++ b/meta/recipes-devtools/python/python-setuptools.inc
>> @@ -9,6 +9,8 @@ PYPI_PACKAGE_EXT = "zip"
>>
>>  inherit pypi
>>
>> +SRC_URI_append_class-native = "
>> file://0001-conditionally-do-not-fetch-code-by-easy_install.patch"
>> +
>>  SRC_URI[md5sum] = "260630ae1a64bafa39dcc53404d63829"
>>  SRC_URI[sha256sum] =
>> "012adb8e25fbfd64c652e99e7bab58799a3aaf05d39ab38561f69190a909015f"
>>
>> --
>> 2.7.4
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20180727/6052c2a6/attachment-0002.html>