[OE-core] [PATCH 1/3] python3: upgrade to 3.7.2

Alejandro Enedino Hernandez Samaniego alejandro.enedino.hernandez-samaniego at xilinx.com
Fri Feb 1 22:32:22 UTC 2019


On 2/1/19 12:49 PM, Khem Raj wrote:
> On Fri, Feb 1, 2019 at 4:59 AM Alexander Kanavin <alex.kanavin at gmail.com> wrote:
>> I took the same approach as the recent perl upgrade: write recipe from scratch,
>> taking the pieces from the old recipe only when they were proven to be necessary.
>>
>> The pgo, manifest and ptest features are all preserved.
>>
>> New features:
>>
>> - native and target recipes are now unified into one recipe
>>
>> - check_build_completeness.py runs right after do_compile() and verifies that
>> all optional modules have been built (a notorious source of regressions)
>>
>> - a new approach to sysconfig.py and distutils/sysconfig.py returning values
>> appropriate for native or target builds: we copy the configuration file to a
>> separate folder, add that folder to sys.path (through environment variable
>> that differs between native and target builds), and point python to the file
>> through another environment variable.
>>
> Thanks, once current cycle is over for OE, I would like to give this a
> whirl and see how it goes
>
>> Signed-off-by: Alexander Kanavin <alex.kanavin at gmail.com>
>> ---
>>   meta/classes/python3-dir.bbclass              |    2 +-
>>   meta/classes/python3native.bbclass            |    2 +
>>   ...ib-termcap-to-linker-flags-to-avoid-.patch |   25 +
>>   ...hell-version-of-python-config-that-w.patch |   35 +
>>   ...-qemu-wrapper-when-gathering-profile.patch |   25 +
>>   ...fig-append-STAGING_LIBDIR-python-sys.patch |   42 +
>>   ...tutils-prefix-is-inside-staging-area.patch |   54 +
>>   .../python3/avoid_warning_about_tkinter.patch |   36 +
>>   .../python-sanity/python3/cgi_py.patch        |   32 +
>>   .../python3/check_build_completeness.py       |   17 +
>>   .../python-sanity/python3/create_manifest3.py |  433 ++++++
>>   .../python-sanity/python3/get_module_deps3.py |  146 ++
>>   .../python-sanity/python3/python-config.patch |   46 +
>>   .../python3/python3-manifest.json             | 1227 +++++++++++++++++
>>   .../python-sanity/python3/run-ptest           |    3 +
>>   .../python-sanity/python3_3.7.2.bb            |  281 ++++
>>   16 files changed, 2405 insertions(+), 1 deletion(-)
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/cgi_py.patch
>>   create mode 100755 meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/create_manifest3.py
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/python-config.patch
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/python3-manifest.json
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3/run-ptest
>>   create mode 100644 meta/recipes-devtools/python-sanity/python3_3.7.2.bb

Thanks Alex!, I looked at this briefly and it looks good.

Do we want to keep the recipe at recipes-devtools-python-sanity?, also I think the PACKAGECONFIG for tk is broken but thats probably tk's fault,
and, since tk is part of meta-oe only, I am wondering if it would make sense to have a bbappend there with the PACKAGECONFIG there,
since we're depending on something that were actually not holding?

Alejandro


>>
>> diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass
>> index 06bb046d9c2..7dd130bad99 100644
>> --- a/meta/classes/python3-dir.bbclass
>> +++ b/meta/classes/python3-dir.bbclass
>> @@ -1,4 +1,4 @@
>> -PYTHON_BASEVERSION = "3.5"
>> +PYTHON_BASEVERSION = "3.7"
>>   PYTHON_ABI = "m"
>>   PYTHON_DIR = "python${PYTHON_BASEVERSION}"
>>   PYTHON_PN = "python3"
>> diff --git a/meta/classes/python3native.bbclass b/meta/classes/python3native.bbclass
>> index da12a714703..a3acaf61bbd 100644
>> --- a/meta/classes/python3native.bbclass
>> +++ b/meta/classes/python3native.bbclass
>> @@ -9,6 +9,8 @@ DEPENDS_append = " python3-native "
>>   export STAGING_INCDIR
>>   export STAGING_LIBDIR
>>
>> +export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>> +
>>   # suppress host user's site-packages dirs.
>>   export PYTHONNOUSERSITE = "1"
>>
>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
>> new file mode 100644
>> index 00000000000..09f279ba1d7
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
>> @@ -0,0 +1,25 @@
>> +From 23294c6ba6896115828293fdb7e67b47b38ba675 Mon Sep 17 00:00:00 2001
>> +From: Alexander Kanavin <alex.kanavin at gmail.com>
>> +Date: Fri, 25 Jan 2019 19:04:13 +0100
>> +Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host
>> + contamination
>> +
>> +Upstream-Status: Inappropriate [oe-core specific]
>> +Signed-off-by: Alexander Kanavin <alex.kanavin at gmail.com>
>> +
>> +---
>> + setup.py | 1 -
>> + 1 file changed, 1 deletion(-)
>> +
>> +diff --git a/setup.py b/setup.py
>> +index b4357e3..fbec00d 100644
>> +--- a/setup.py
>> ++++ b/setup.py
>> +@@ -856,7 +856,6 @@ class PyBuildExt(build_ext):
>> +                                                      'termcap'):
>> +                 readline_libs.append('termcap')
>> +             exts.append( Extension('readline', ['readline.c'],
>> +-                                   library_dirs=['/usr/lib/termcap'],
>> +                                    extra_link_args=readline_extra_link_args,
>> +                                    libraries=readline_libs) )
>> +         else:
>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
>> new file mode 100644
>> index 00000000000..83fd52d87f4
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
>> @@ -0,0 +1,35 @@
>> +From 148861fa16f2aaacd518770f337ea54b5182f981 Mon Sep 17 00:00:00 2001
>> +From: Alexander Kanavin <alex.kanavin at gmail.com>
>> +Date: Tue, 29 Jan 2019 15:03:01 +0100
>> +Subject: [PATCH] Do not use the shell version of python-config that was
>> + introduced in 3.4
>> +
>> +Revert instead to the original python version: it has our tweaks and
>> +outputs directories correctly.
>> +
>> +Upstream-Status: Inappropriate [oe-specific]
>> +Signed-off-by: Alexander Kanavin <alex.kanavin at gmail.com>
>> +---
>> + Makefile.pre.in | 9 +++------
>> + 1 file changed, 3 insertions(+), 6 deletions(-)
>> +
>> +diff --git a/Makefile.pre.in b/Makefile.pre.in
>> +index 2d2e11f..cc19942 100644
>> +--- a/Makefile.pre.in
>> ++++ b/Makefile.pre.in
>> +@@ -1431,12 +1431,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
>> +       sed -e "s, at EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
>> +       @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
>> +       LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
>> +-      @ # On Darwin, always use the python version of the script, the shell
>> +-      @ # version doesn't use the compiler customizations that are provided
>> +-      @ # in python (_osx_support.py).
>> +-      @if test `uname -s` = Darwin; then \
>> +-              cp python-config.py python-config; \
>> +-      fi
>> ++      @  # In OpenEmbedded, always use the python version of the script, the shell
>> ++      @  # version is broken in multiple ways, and doesn't return correct directories
>> ++      cp python-config.py python-config
>> +
>> +
>> + # Install the include files
>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
>> new file mode 100644
>> index 00000000000..fa7735ff93e
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
>> @@ -0,0 +1,25 @@
>> +From cf6a9100902484e4d028ee88742dd2487b014a98 Mon Sep 17 00:00:00 2001
>> +From: Alexander Kanavin <alex.kanavin at gmail.com>
>> +Date: Wed, 30 Jan 2019 12:41:04 +0100
>> +Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data
>> +
>> +Upstream-Status: Inappropriate [oe-core specific]
>> +Signed-off-by: Alexander Kanavin <alex.kanavin at gmail.com>
>> +---
>> + Makefile.pre.in | 3 +--
>> + 1 file changed, 1 insertion(+), 2 deletions(-)
>> +
>> +diff --git a/Makefile.pre.in b/Makefile.pre.in
>> +index a3a02a7..d5503dd 100644
>> +--- a/Makefile.pre.in
>> ++++ b/Makefile.pre.in
>> +@@ -507,8 +507,7 @@ build_all_generate_profile:
>> +       $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)"
>> +
>> + run_profile_task:
>> +-      @ # FIXME: can't run for a cross build
>> +-      $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
>> ++      ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true
>> +
>> + build_all_merge_profile:
>> +       $(LLVM_PROF_MERGER)
>> diff --git a/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
>> new file mode 100644
>> index 00000000000..2c5b76f131c
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
>> @@ -0,0 +1,42 @@
>> +From 5464c6f3fe2d3e3854f7798838aa550207f4b417 Mon Sep 17 00:00:00 2001
>> +From: Alexander Kanavin <alex.kanavin at gmail.com>
>> +Date: Thu, 31 Jan 2019 16:46:30 +0100
>> +Subject: [PATCH] distutils/sysconfig: append
>> + STAGING_LIBDIR/python-sysconfigdata to sys.path
>> +
>> +So that target configuration can be used when running native python
>> +
>> +Upstream-Status: Inappropriate [oe-core specific]
>> +Signed-off-by: Alexander Kanavin <alex.kanavin at gmail.com>
>> +
>> +---
>> + Lib/distutils/sysconfig.py | 2 ++
>> + Lib/sysconfig.py           | 2 ++
>> + 2 files changed, 4 insertions(+)
>> +
>> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
>> +index e07a6c8..6b8c129 100644
>> +--- a/Lib/distutils/sysconfig.py
>> ++++ b/Lib/distutils/sysconfig.py
>> +@@ -421,6 +421,8 @@ def _init_posix():
>> +         platform=sys.platform,
>> +         multiarch=getattr(sys.implementation, '_multiarch', ''),
>> +     ))
>> ++    if 'STAGING_LIBDIR' in os.environ:
>> ++        sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
>> +     _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
>> +     build_time_vars = _temp.build_time_vars
>> +     global _config_vars
>> +diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
>> +index 9ee4d31..e586abd 100644
>> +--- a/Lib/sysconfig.py
>> ++++ b/Lib/sysconfig.py
>> +@@ -412,6 +412,8 @@ def _init_posix(vars):
>> +     """Initialize the module as appropriate for POSIX systems."""
>> +     # _sysconfigdata is generated at build time, see _generate_posix_vars()
>> +     name = _get_sysconfigdata_name()
>> ++    if 'STAGING_LIBDIR' in os.environ:
>> ++        sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
>> +     _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
>> +     build_time_vars = _temp.build_time_vars
>> +     vars.update(build_time_vars)
>> diff --git a/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
>> new file mode 100644
>> index 00000000000..abf08f59dd7
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/12-distutils-prefix-is-inside-staging-area.patch
>> @@ -0,0 +1,54 @@
>> +From fc27c603d7e9efea987a9764b524427990cbf1f1 Mon Sep 17 00:00:00 2001
>> +From: Khem Raj <raj.khem at gmail.com>
>> +Date: Tue, 14 May 2013 15:00:26 -0700
>> +Subject: [PATCH] python3: Add target and native recipes
>> +
>> +Upstream-Status: Inappropriate [embedded specific]
>> +
>> +02/2015 Rebased for Python 3.4.2
>> +
>> +# The proper prefix is inside our staging area.
>> +# Signed-Off: Michael 'Mickey' Lauer <mickey at vanille-media.de>
>> +# Signed-off-by: Phil Blundell <philb at gnu.org>
>> +# Signed-off-by: Khem Raj <raj.khem at gmail.com>
>> +# Signed-off-by: Alejandro Hernandez <alejandro.hernandez at linux.intel.com>
>> +
>> +---
>> + Lib/distutils/sysconfig.py | 10 ++++++++--
>> + 1 file changed, 8 insertions(+), 2 deletions(-)
>> +
>> +diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
>> +index 6b8c129..3ca7f79 100644
>> +--- a/Lib/distutils/sysconfig.py
>> ++++ b/Lib/distutils/sysconfig.py
>> +@@ -84,7 +84,9 @@ def get_python_inc(plat_specific=0, prefix=None):
>> +     If 'prefix' is supplied, use it instead of sys.base_prefix or
>> +     sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
>> +     """
>> +-    if prefix is None:
>> ++    if prefix is None and os.environ['STAGING_INCDIR'] != "":
>> ++        prefix = os.environ['STAGING_INCDIR'].rstrip('include')
>> ++    elif prefix is None:
>> +         prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
>> +     if os.name == "posix":
>> +         if python_build:
>> +@@ -122,6 +124,10 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
>> +     If 'prefix' is supplied, use it instead of sys.base_prefix or
>> +     sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
>> +     """
>> ++    lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1]
>> ++    if prefix is None and os.environ['STAGING_LIBDIR'] != "":
>> ++        prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename)
>> ++
>> +     if prefix is None:
>> +         if standard_lib:
>> +             prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
>> +@@ -130,7 +136,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
>> +
>> +     if os.name == "posix":
>> +         libpython = os.path.join(prefix,
>> +-                                 "lib", "python" + get_python_version())
>> ++                                 lib_basename, "python" + get_python_version())
>> +         if standard_lib:
>> +             return libpython
>> +         else:
>> diff --git a/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
>> new file mode 100644
>> index 00000000000..24e67b4ca14
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/avoid_warning_about_tkinter.patch
>> @@ -0,0 +1,36 @@
>> +From fead48c8b501a8d7c3db21df2e599f90f38f11d3 Mon Sep 17 00:00:00 2001
>> +From: Andrei Gherzan <andrei at gherzan.ro>
>> +Date: Mon, 28 Jan 2019 15:57:54 +0000
>> +Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet
>> + integrated in yocto so we skip the check for this module. Avoid a warning by
>> + not adding this module to missing variable.
>> +
>> +Upstream-Status: Inappropriate [distribution]
>> +
>> +Also simply disable the tk module since its not in DEPENDS.
>> +Signed-off-by: Andrei Gherzan <andrei at gherzan.ro>
>> +
>> +---
>> + setup.py | 8 +++++---
>> + 1 file changed, 5 insertions(+), 3 deletions(-)
>> +
>> +diff --git a/setup.py b/setup.py
>> +index fbec00d..b7a36a6 100644
>> +--- a/setup.py
>> ++++ b/setup.py
>> +@@ -1623,10 +1623,12 @@ class PyBuildExt(build_ext):
>> +         self.extensions.extend(exts)
>> +
>> +         # Call the method for detecting whether _tkinter can be compiled
>> +-        self.detect_tkinter(inc_dirs, lib_dirs)
>> ++        # self.detect_tkinter(inc_dirs, lib_dirs)
>> +
>> +-        if '_tkinter' not in [e.name for e in self.extensions]:
>> +-            missing.append('_tkinter')
>> ++        # tkinter module will not be avalaible as yocto
>> ++        # doesn't have tk integrated (yet)
>> ++        #if '_tkinter' not in [e.name for e in self.extensions]:
>> ++        #    missing.append('_tkinter')
>> +
>> +         # Build the _uuid module if possible
>> +         uuid_incs = find_file("uuid.h", inc_dirs, ["/usr/include/uuid"])
>> diff --git a/meta/recipes-devtools/python-sanity/python3/cgi_py.patch b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch
>> new file mode 100644
>> index 00000000000..6c4ba54320b
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/cgi_py.patch
>> @@ -0,0 +1,32 @@
>> +From 62336285cba38017b35cb761c03f0c7e80a671a3 Mon Sep 17 00:00:00 2001
>> +From: Mark Hatle <mark.hatle at windriver.com>
>> +Date: Wed, 21 Sep 2011 20:55:33 -0500
>> +Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment
>> +
>> +Upstream-Status: Inappropriate [distribution]
>> +
>> +Signed-off-by: Mark Hatle <mark.hatle at windriver.com>
>> +
>> +---
>> + Lib/cgi.py | 11 +----------
>> + 1 file changed, 1 insertion(+), 10 deletions(-)
>> +
>> +diff --git a/Lib/cgi.py b/Lib/cgi.py
>> +index 8cf6687..094c7b4 100755
>> +--- a/Lib/cgi.py
>> ++++ b/Lib/cgi.py
>> +@@ -1,13 +1,4 @@
>> +-#! /usr/local/bin/python
>> +-
>> +-# NOTE: the above "/usr/local/bin/python" is NOT a mistake.  It is
>> +-# intentionally NOT "/usr/bin/env python".  On many systems
>> +-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI
>> +-# scripts, and /usr/local/bin is the default directory where Python is
>> +-# installed, so /usr/bin/env would be unable to find python.  Granted,
>> +-# binary installations by Linux vendors often install Python in
>> +-# /usr/bin.  So let those vendors patch cgi.py to match their choice
>> +-# of installation.
>> ++#! /usr/bin/env python
>> +
>> + """Support module for CGI (Common Gateway Interface) scripts.
>> +
>> diff --git a/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
>> new file mode 100755
>> index 00000000000..a1eace3f571
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/check_build_completeness.py
>> @@ -0,0 +1,17 @@
>> +#!/usr/bin/env python3
>> +import sys
>> +logfile = open(sys.argv[1]).read()
>> +
>> +necessary_bits = logfile.find("The necessary bits to build these optional modules were not found")
>> +to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.")
>> +if necessary_bits != -1:
>> +    print("%s" %(logfile[necessary_bits:to_find_bits]))
>> +
>> +failed_to_build = logfile.find("Failed to build these modules:")
>> +if failed_to_build != -1:
>> +    failed_to_build_end = logfile.find("\n\n", failed_to_build)
>> +    print("%s" %(logfile[failed_to_build:failed_to_build_end]))
>> +
>> +if necessary_bits != -1 or failed_to_build != -1:
>> +    sys.exit(1)
>> +
>> diff --git a/meta/recipes-devtools/python-sanity/python3/create_manifest3.py b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py
>> new file mode 100644
>> index 00000000000..4da02a2991a
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/create_manifest3.py
>> @@ -0,0 +1,433 @@
>> +# This script is used as a bitbake task to create a new python manifest
>> +# $ bitbake python -c create_manifest
>> +#
>> +# Our goal is to keep python-core as small as posible and add other python
>> +# packages only when the user needs them, hence why we split upstream python
>> +# into several packages.
>> +#
>> +# In a very simplistic way what this does is:
>> +# Launch python and see specifically what is required for it to run at a minimum
>> +#
>> +# Go through the python-manifest file and launch a separate task for every single
>> +# one of the files on each package, this task will check what was required for that
>> +# specific module to run, these modules will be called dependencies.
>> +# The output of such task will be a list of the modules or dependencies that were
>> +# found for that file.
>> +#
>> +# Such output will be parsed by this script, we will look for each dependency on the
>> +# manifest and if we find that another package already includes it, then we will add
>> +# that package as an RDEPENDS to the package we are currently checking; in case we dont
>> +# find the current dependency on any other package we will add it to the current package
>> +# as part of FILES.
>> +#
>> +#
>> +# This way we will create a new manifest from the data structure that was built during
>> +# this process, on this new manifest each package will contain specifically only
>> +# what it needs to run.
>> +#
>> +# There are some caveats which we try to deal with, such as repeated files on different
>> +# packages, packages that include folders, wildcards, and special packages.
>> +# Its also important to note that this method only works for python files, and shared
>> +# libraries. Static libraries, header files and binaries need to be dealt with manually.
>> +#
>> +# This script differs from its python2 version mostly on how shared libraries are handled
>> +# The manifest file for python3 has an extra field which contains the cached files for
>> +# each package.
>> +# Tha method to handle cached files does not work when a module includes a folder which
>> +# itself contains the pycache folder, gladly this is almost never the case.
>> +#
>> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com>
>> +
>> +
>> +import sys
>> +import subprocess
>> +import json
>> +import os
>> +import collections
>> +
>> +# Get python version from ${PYTHON_MAJMIN}
>> +pyversion = str(sys.argv[1])
>> +
>> +# Hack to get native python search path (for folders), not fond of it but it works for now
>> +pivot = 'recipe-sysroot-native'
>> +for p in sys.path:
>> +    if pivot in p:
>> +        nativelibfolder = p[:p.find(pivot)+len(pivot)]
>> +
>> +# Empty dict to hold the whole manifest
>> +new_manifest = collections.OrderedDict()
>> +
>> +# Check for repeated files, folders and wildcards
>> +allfiles = []
>> +repeated = []
>> +wildcards = []
>> +
>> +hasfolders = []
>> +allfolders = []
>> +
>> +def isFolder(value):
>> +    value = value.replace('${PYTHON_MAJMIN}',pyversion)
>> +    if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')):
>> +        return True
>> +    else:
>> +        return False
>> +
>> +def isCached(item):
>> +    if '__pycache__' in item:
>> +        return True
>> +    else:
>> +        return False
>> +
>> +def prepend_comments(comments, json_manifest):
>> +    with open(json_manifest, 'r+') as manifest:
>> +        json_contents = manifest.read()
>> +        manifest.seek(0, 0)
>> +        manifest.write(comments + json_contents)
>> +
>> +# Read existing JSON manifest
>> +with open('python3-manifest.json') as manifest:
>> +    # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker
>> +    manifest_str =  manifest.read()
>> +    json_start = manifest_str.find('# EOC') + 6 # EOC + \n
>> +    manifest.seek(0)
>> +    comments = manifest.read(json_start)
>> +    manifest_str = manifest.read()
>> +    old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict)
>> +
>> +#
>> +# First pass to get core-package functionality, because we base everything on the fact that core is actually working
>> +# Not exactly the same so it should not be a function
>> +#
>> +
>> +print ('Getting dependencies for package: core')
>> +
>> +
>> +# This special call gets the core dependencies and
>> +# appends to the old manifest so it doesnt hurt what it
>> +# currently holds.
>> +# This way when other packages check for dependencies
>> +# on the new core package, they will still find them
>> +# even when checking the old_manifest
>> +
>> +output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8')
>> +for coredep in output.split():
>> +    coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}')
>> +    if isCached(coredep):
>> +        if coredep not in old_manifest['core']['cached']:
>> +            old_manifest['core']['cached'].append(coredep)
>> +    else:
>> +        if coredep not in old_manifest['core']['files']:
>> +            old_manifest['core']['files'].append(coredep)
>> +
>> +
>> +# The second step is to loop through the existing files contained in the core package
>> +# according to the old manifest, identify if they are  modules, or some other type
>> +# of file that we cant import (directories, binaries, configs) in which case we
>> +# can only assume they were added correctly (manually) so we ignore those and
>> +# pass them to the manifest directly.
>> +
>> +for filedep in old_manifest['core']['files']:
>> +    if isFolder(filedep):
>> +        if isCached(filedep):
>> +            if filedep not in old_manifest['core']['cached']:
>> +                old_manifest['core']['cached'].append(filedep)
>> +        else:
>> +            if filedep not in old_manifest['core']['files']:
>> +                old_manifest['core']['files'].append(filedep)
>> +        continue
>> +    if '${bindir}' in filedep:
>> +        if filedep not in old_manifest['core']['files']:
>> +            old_manifest['core']['files'].append(filedep)
>> +        continue
>> +    if filedep == '':
>> +        continue
>> +    if '${includedir}' in filedep:
>> +        if filedep not in old_manifest['core']['files']:
>> +            old_manifest['core']['files'].append(filedep)
>> +        continue
>> +
>> +    # Get actual module name , shouldnt be affected by libdir/bindir, etc.
>> +    pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0]
>> +
>> +
>> +    # We now know that were dealing with a python module, so we can import it
>> +    # and check what its dependencies are.
>> +    # We launch a separate task for each module for deterministic behavior.
>> +    # Each module will only import what is necessary for it to work in specific.
>> +    # The output of each task will contain each module's dependencies
>> +
>> +    print ('Getting dependencies for module: %s' % pymodule)
>> +    output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8')
>> +    print ('The following dependencies were found for module %s:\n' % pymodule)
>> +    print (output)
>> +
>> +
>> +    for pymodule_dep in output.split():
>> +        pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
>> +
>> +        if isCached(pymodule_dep):
>> +            if pymodule_dep not in old_manifest['core']['cached']:
>> +                old_manifest['core']['cached'].append(pymodule_dep)
>> +        else:
>> +            if pymodule_dep not in old_manifest['core']['files']:
>> +                old_manifest['core']['files'].append(pymodule_dep)
>> +
>> +
>> +# At this point we are done with the core package.
>> +# The old_manifest dictionary is updated only for the core package because
>> +# all others will use this a base.
>> +
>> +
>> +# To improve the script speed, we check which packages contain directories
>> +# since we will be looping through (only) those later.
>> +for pypkg in old_manifest:
>> +    for filedep in old_manifest[pypkg]['files']:
>> +        if isFolder(filedep):
>> +            print ('%s is a folder' % filedep)
>> +            if pypkg not in hasfolders:
>> +                hasfolders.append(pypkg)
>> +            if filedep not in allfolders:
>> +                allfolders.append(filedep)
>> +
>> +
>> +
>> +# This is the main loop that will handle each package.
>> +# It works in a similar fashion than the step before, but
>> +# we will now be updating a new dictionary that will eventually
>> +# become the new manifest.
>> +#
>> +# The following loops though all packages in the manifest,
>> +# through all files on each of them, and checks whether or not
>> +# they are modules and can be imported.
>> +# If they can be imported, then it checks for dependencies for
>> +# each of them by launching a separate task.
>> +# The output of that task is then parsed and the manifest is updated
>> +# accordingly, wether it should add the module on FILES for the current package
>> +# or if that module already belongs to another package then the current one
>> +# will RDEPEND on it
>> +
>> +for pypkg in old_manifest:
>> +    # Use an empty dict as data structure to hold data for each package and fill it up
>> +    new_manifest[pypkg] = collections.OrderedDict()
>> +    new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary']
>> +    new_manifest[pypkg]['rdepends'] = []
>> +    new_manifest[pypkg]['files'] = []
>> +    new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached']
>> +
>> +    # All packages should depend on core
>> +    if pypkg != 'core':
>> +        new_manifest[pypkg]['rdepends'].append('core')
>> +        new_manifest[pypkg]['cached'] = []
>> +
>> +    print('\n')
>> +    print('--------------------------')
>> +    print ('Handling package %s' % pypkg)
>> +    print('--------------------------')
>> +
>> +    # Handle special cases, we assume that when they were manually added
>> +    # to the manifest we knew what we were doing.
>> +    special_packages = ['misc', 'modules', 'dev', 'tests']
>> +    if pypkg in special_packages or 'staticdev' in pypkg:
>> +        print('Passing %s package directly' % pypkg)
>> +        new_manifest[pypkg] = old_manifest[pypkg]
>> +        continue
>> +
>> +    for filedep in old_manifest[pypkg]['files']:
>> +        # We already handled core on the first pass, we can ignore it now
>> +        if pypkg == 'core':
>> +            if filedep not in new_manifest[pypkg]['files']:
>> +                new_manifest[pypkg]['files'].append(filedep)
>> +            continue
>> +
>> +        # Handle/ignore what we cant import
>> +        if isFolder(filedep):
>> +            new_manifest[pypkg]['files'].append(filedep)
>> +            # Asyncio (and others) are both the package and the folder name, we should not skip those...
>> +            path,mod = os.path.split(filedep)
>> +            if mod != pypkg:
>> +                continue
>> +        if '${bindir}' in filedep:
>> +            if filedep not in new_manifest[pypkg]['files']:
>> +                new_manifest[pypkg]['files'].append(filedep)
>> +            continue
>> +        if filedep == '':
>> +            continue
>> +        if '${includedir}' in filedep:
>> +            if filedep not in new_manifest[pypkg]['files']:
>> +                new_manifest[pypkg]['files'].append(filedep)
>> +            continue
>> +
>> +        # Get actual module name , shouldnt be affected by libdir/bindir, etc.
>> +        # We need to check if the imported module comes from another (e.g. sqlite3.dump)
>> +        path,pymodule = os.path.split(filedep)
>> +        path = os.path.basename(path)
>> +        pymodule = os.path.splitext(os.path.basename(pymodule))[0]
>> +
>> +        # If this condition is met, it means we need to import it from another module
>> +        # or its the folder itself (e.g. unittest)
>> +        if path == pypkg:
>> +            if pymodule:
>> +                pymodule = path + '.' + pymodule
>> +            else:
>> +                pymodule = path
>> +
>> +
>> +
>> +        # We now know that were dealing with a python module, so we can import it
>> +        # and check what its dependencies are.
>> +        # We launch a separate task for each module for deterministic behavior.
>> +        # Each module will only import what is necessary for it to work in specific.
>> +        # The output of each task will contain each module's dependencies
>> +
>> +        print ('\nGetting dependencies for module: %s' % pymodule)
>> +        output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8')
>> +        print ('The following dependencies were found for module %s:\n' % pymodule)
>> +        print (output)
>> +
>> +        reportFILES = []
>> +        reportRDEPS = []
>> +
>> +        for pymodule_dep in output.split():
>> +
>> +            # Warning: This first part is ugly
>> +            # One of the dependencies that was found, could be inside of one of the folders included by another package
>> +            # We need to check if this happens so we can add the package containing the folder as an rdependency
>> +            # e.g. Folder encodings contained in codecs
>> +            # This would be solved if no packages included any folders
>> +
>> +            # This can be done in two ways:
>> +            # 1 - We assume that if we take out the filename from the path we would get
>> +            #   the folder string, then we would check if folder string is in the list of folders
>> +            #   This would not work if a package contains a folder which contains another folder
>> +            #   e.g. path/folder1/folder2/filename  folder_string= path/folder1/folder2
>> +            #   folder_string would not match any value contained in the list of folders
>> +            #
>> +            # 2 - We do it the other way around, checking if the folder is contained in the path
>> +            #   e.g. path/folder1/folder2/filename  folder_string= path/folder1/folder2
>> +            #   is folder_string inside path/folder1/folder2/filename?,
>> +            #   Yes, it works, but we waste a couple of milliseconds.
>> +
>> +            pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
>> +            inFolders = False
>> +            for folder in allfolders:
>> +                # The module could have a directory named after it, e.g. xml, if we take out the filename from the path
>> +                # we'll end up with ${libdir}, and we want ${libdir}/xml
>> +                if isFolder(pymodule_dep):
>> +                    check_path = pymodule_dep
>> +                else:
>> +                    check_path = os.path.dirname(pymodule_dep)
>> +                if folder in check_path :
>> +                    inFolders = True # Did we find a folder?
>> +                    folderFound = False # Second flag to break inner for
>> +                    # Loop only through packages which contain folders
>> +                    for pypkg_with_folder in hasfolders:
>> +                        if (folderFound == False):
>> +                            # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder))
>> +                            for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']:
>> +                                if folder_dep == folder:
>> +                                    print ('%s folder found in %s' % (folder, pypkg_with_folder))
>> +                                    folderFound = True
>> +                                    if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg:
>> +                                        new_manifest[pypkg]['rdepends'].append(pypkg_with_folder)
>> +                        else:
>> +                            break
>> +
>> +            # A folder was found so we're done with this item, we can go on
>> +            if inFolders:
>> +                continue
>> +
>> +
>> +
>> +            # No directories beyond this point
>> +            # We might already have this module on the dictionary since it could depend on a (previously checked) module
>> +            if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']:
>> +                # Handle core as a special package, we already did it so we pass it to NEW data structure directly
>> +                if pypkg == 'core':
>> +                    print('Adding %s to %s FILES' % (pymodule_dep, pypkg))
>> +                    if pymodule_dep.endswith('*'):
>> +                        wildcards.append(pymodule_dep)
>> +                    if isCached(pymodule_dep):
>> +                        new_manifest[pypkg]['cached'].append(pymodule_dep)
>> +                    else:
>> +                        new_manifest[pypkg]['files'].append(pymodule_dep)
>> +
>> +                    # Check for repeated files
>> +                    if pymodule_dep not in allfiles:
>> +                        allfiles.append(pymodule_dep)
>> +                    else:
>> +                        if pymodule_dep not in repeated:
>> +                            repeated.append(pymodule_dep)
>> +                else:
>> +
>> +
>> +                    # Last step: Figure out if we this belongs to FILES or RDEPENDS
>> +                    # We check if this module is already contained on another package, so we add that one
>> +                    # as an RDEPENDS, or if its not, it means it should be contained on the current
>> +                    # package, and we should add it to FILES
>> +                    for possible_rdep in old_manifest:
>> +                        # Debug
>> +                        # print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep)
>> +                        if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']:
>> +                            # Since were nesting, we need to check its not the same pypkg
>> +                            if(possible_rdep != pypkg):
>> +                                if possible_rdep not in new_manifest[pypkg]['rdepends']:
>> +                                    # Add it to the new manifest data struct as RDEPENDS since it contains something this module needs
>> +                                    reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep))
>> +                                    new_manifest[pypkg]['rdepends'].append(possible_rdep)
>> +                                break
>> +                    else:
>> +
>> +                      # Since this module wasnt found on another package, it is not an RDEP,
>> +                      # so we add it to FILES for this package.
>> +                      # A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files)
>> +                      if os.path.basename(pymodule_dep) != pypkg:
>> +                        reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg)))
>> +                        if isCached(pymodule_dep):
>> +                            new_manifest[pypkg]['cached'].append(pymodule_dep)
>> +                        else:
>> +                            new_manifest[pypkg]['files'].append(pymodule_dep)
>> +                        if pymodule_dep.endswith('*'):
>> +                            wildcards.append(pymodule_dep)
>> +                        if pymodule_dep not in allfiles:
>> +                            allfiles.append(pymodule_dep)
>> +                        else:
>> +                            if pymodule_dep not in repeated:
>> +                                repeated.append(pymodule_dep)
>> +
>> +        print('\n')
>> +        print('#################################')
>> +        print('Summary for module %s' % pymodule)
>> +        print('FILES found for module %s:' % pymodule)
>> +        print(''.join(reportFILES))
>> +        print('RDEPENDS found for module %s:' % pymodule)
>> +        print(''.join(reportRDEPS))
>> +        print('#################################')
>> +
>> +print('The following FILES contain wildcards, please check if they are necessary')
>> +print(wildcards)
>> +print('The following FILES contain folders, please check if they are necessary')
>> +print(hasfolders)
>> +
>> +
>> +# Sort it just so it looks nicer
>> +for pypkg in new_manifest:
>> +    new_manifest[pypkg]['files'].sort()
>> +    new_manifest[pypkg]['cached'].sort()
>> +    new_manifest[pypkg]['rdepends'].sort()
>> +
>> +# Create the manifest from the data structure that was built
>> +with open('python3-manifest.json.new','w') as outfile:
>> +    json.dump(new_manifest,outfile, indent=4)
>> +    outfile.write('\n')
>> +
>> +prepend_comments(comments,'python3-manifest.json.new')
>> +
>> +if (repeated):
>> +    error_msg = '\n\nERROR:\n'
>> +    error_msg += 'The following files are repeated (contained in more than one package),\n'
>> +    error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n'
>> +    error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n'
>> +    error_msg += '\n'.join(repeated)
>> +    error_msg += '\n'
>> +    sys.exit(error_msg)
>> +
>> diff --git a/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
>> new file mode 100644
>> index 00000000000..fd12baad84e
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/get_module_deps3.py
>> @@ -0,0 +1,146 @@
>> +# This script is launched on separate task for each python module
>> +# It checks for dependencies for that specific module and prints
>> +# them out, the output of this execution will have all dependencies
>> +# for a specific module, which will be parsed an dealt on create_manifest.py
>> +#
>> +# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail.com>
>> +
>> +# We can get a log per module, for all the dependencies that were found, but its messy.
>> +debug=False
>> +
>> +import sys
>> +
>> +# We can get a list of the modules which are currently required to run python
>> +# so we run python-core and get its modules, we then import what we need
>> +# and check what modules are currently running, if we substract them from the
>> +# modules we had initially, we get the dependencies for the module we imported.
>> +
>> +# We use importlib to achieve this, so we also need to know what modules importlib needs
>> +import importlib
>> +
>> +core_deps=set(sys.modules)
>> +
>> +def fix_path(dep_path):
>> +    import os
>> +    # We DONT want the path on our HOST system
>> +    pivot='recipe-sysroot-native'
>> +    dep_path=dep_path[dep_path.find(pivot)+len(pivot):]
>> +
>> +    if '/usr/bin' in dep_path:
>> +        dep_path = dep_path.replace('/usr/bin''${bindir}')
>> +
>> +    # Handle multilib, is there a better way?
>> +    if '/usr/lib32' in dep_path:
>> +        dep_path = dep_path.replace('/usr/lib32','${libdir}')
>> +    if '/usr/lib64' in dep_path:
>> +        dep_path = dep_path.replace('/usr/lib64','${libdir}')
>> +    if '/usr/lib' in dep_path:
>> +        dep_path = dep_path.replace('/usr/lib','${libdir}')
>> +    if '/usr/include' in dep_path:
>> +        dep_path = dep_path.replace('/usr/include','${includedir}')
>> +    if '__init__.' in dep_path:
>> +        dep_path =  os.path.split(dep_path)[0]
>> +    return dep_path
>> +
>> +
>> +# Module to import was passed as an argument
>> +current_module =  str(sys.argv[1]).rstrip()
>> +if(debug==True):
>> +    log = open('log_%s' % current_module,'w')
>> +    log.write('Module %s generated the following dependencies:\n' % current_module)
>> +try:
>> +    importlib.import_module('%s' % current_module)
>> +except ImportError as e:
>> +    if (debug==True):
>> +        log.write('Module was not found')
>> +    pass
>> +
>> +
>> +# Get current module dependencies, dif will contain a list of specific deps for this module
>> +module_deps=set(sys.modules)
>> +
>> +# We handle the core package (1st pass on create_manifest.py) as a special case
>> +if current_module == 'python-core-package':
>> +    dif = core_deps
>> +else:
>> +    # We know this is not the core package, so there must be a difference.
>> +    dif = module_deps-core_deps
>> +
>> +
>> +# Check where each dependency came from
>> +for item in dif:
>> +    dep_path=''
>> +    try:
>> +        if (debug==True):
>> +            log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n')
>> +        dep_path = sys.modules['%s' % item].__file__
>> +    except AttributeError as e:
>> +        # Deals with thread (builtin module) not having __file__ attribute
>> +        if debug==True:
>> +            log.write(item + ' ')
>> +            log.write(str(e))
>> +            log.write('\n')
>> +        pass
>> +    except NameError as e:
>> +        # Deals with NameError: name 'dep_path' is not defined
>> +        # because module is not found (wasn't compiled?), e.g. bddsm
>> +        if (debug==True):
>> +            log.write(item+' ')
>> +            log.write(str(e))
>> +        pass
>> +
>> +    # Site-customize is a special case since we (OpenEmbedded) put it there manually
>> +    if 'sitecustomize' in dep_path:
>> +        dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py'
>> +        # Prints out result, which is what will be used by create_manifest
>> +        print (dep_path)
>> +        continue
>> +
>> +    dep_path = fix_path(dep_path)
>> +
>> +    import sysconfig
>> +    soabi=sysconfig.get_config_var('SOABI')
>> +    # Check if its a shared library and deconstruct it
>> +    if soabi in dep_path:
>> +        if (debug==True):
>> +            log.write('Shared library found in %s' % dep_path)
>> +        dep_path = dep_path.replace(soabi,'*')
>> +        print (dep_path)
>> +        continue
>> +
>> +    if (debug==True):
>> +        log.write(dep_path+'\n')
>> +    # Prints out result, which is what will be used by create_manifest
>> +    print (dep_path)
>> +
>> +
>> +    import imp
>> +    cpython_tag = imp.get_tag()
>> +    cached=''
>> +    # Theres no naive way to find *.pyc files on python3
>> +    try:
>> +        if (debug==True):
>> +            log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n')
>> +        cached = sys.modules['%s' % item].__cached__
>> +    except AttributeError as e:
>> +        # Deals with thread (builtin module) not having __cached__ attribute
>> +        if debug==True:
>> +            log.write(item + ' ')
>> +            log.write(str(e))
>> +            log.write('\n')
>> +        pass
>> +    except NameError as e:
>> +        # Deals with NameError: name 'cached' is not defined
>> +        if (debug==True):
>> +            log.write(item+' ')
>> +            log.write(str(e))
>> +        pass
>> +    if cached is not None:
>> +        if (debug==True):
>> +            log.write(cached)
>> +        cached = fix_path(cached)
>> +        cached = cached.replace(cpython_tag,'*')
>> +        print (cached)
>> +
>> +if debug==True:
>> +    log.close()
>> diff --git a/meta/recipes-devtools/python-sanity/python3/python-config.patch b/meta/recipes-devtools/python-sanity/python3/python-config.patch
>> new file mode 100644
>> index 00000000000..f23b8b7df06
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/python-config.patch
>> @@ -0,0 +1,46 @@
>> +python-config: Revert to using distutils.sysconfig
>> +
>> +The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in
>> +
>> +12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig
>> +
>> +affect the native runtime as well as cross building. Use the old, patched
>> +implementation which returns paths in the staging directory and for the target,
>> +as appropriate.
>> +
>> +Upstream-Status: Inappropriate [Embedded Specific]
>> +
>> +Signed-off-by: Tyler Hall <tylerwhall at gmail.com>
>> +:
>> +Index: Python-3.3.3/Misc/python-config.in
>> +===================================================================
>> +--- Python-3.3.3.orig/Misc/python-config.in
>> ++++ Python-3.3.3/Misc/python-config.in
>> +@@ -4,7 +4,7 @@
>> + import getopt
>> + import os
>> + import sys
>> +-import sysconfig
>> ++from distutils import sysconfig
>> +
>> + valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags',
>> +               'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir']
>> +@@ -32,14 +32,14 @@ if '--help' in opt_flags:
>> +
>> + for opt in opt_flags:
>> +     if opt == '--prefix':
>> +-        print(sysconfig.get_config_var('prefix'))
>> ++        print(sysconfig.PREFIX)
>> +
>> +     elif opt == '--exec-prefix':
>> +-        print(sysconfig.get_config_var('exec_prefix'))
>> ++        print(sysconfig.EXEC_PREFIX)
>> +
>> +     elif opt in ('--includes', '--cflags'):
>> +-        flags = ['-I' + sysconfig.get_path('include'),
>> +-                 '-I' + sysconfig.get_path('platinclude')]
>> ++        flags = ['-I' + sysconfig.get_python_inc(),
>> ++                 '-I' + sysconfig.get_python_inc(plat_specific=True)]
>> +         if opt == '--cflags':
>> +             flags.extend(getvar('CFLAGS').split())
>> +         print(' '.join(flags))
>> diff --git a/meta/recipes-devtools/python-sanity/python3/python3-manifest.json b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
>> new file mode 100644
>> index 00000000000..82c7075f0e7
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/python3-manifest.json
>> @@ -0,0 +1,1227 @@
>> +# DO NOT (entirely) modify this file manually, please read.
>> +#
>> +# IMPORTANT NOTE:
>> +# Please keep in mind that the create_manifest task relies on the fact the the
>> +# target and native Python packages are the same, and it also needs to be executed
>> +# with a fully working native package (with all the PACKAGECONFIGs enabled and all
>> +# and all the modules should be working, check log.do_compile), otherwise the script
>> +# will fail to find dependencies correctly, this note is valid either if you are
>> +# upgrading to a new Python version or adding a new package.
>> +#
>> +#
>> +# If you are adding a new package please follow the next steps:
>> +#     How to add a new package:
>> +#     - If a user wants to add a new package all that has to be done is:
>> +#     Modify the python3-manifest.json file, and add the required file(s) to the FILES list,
>> +#     fill up the SUMMARY section as well, the script should handle all the rest.
>> +#
>> +#     Real example:
>> +#     We want to add a web browser package, including the file webbrowser.py
>> +#     which at the moment is on python3-misc.
>> +#     "webbrowser": {
>> +#         "files": ["${libdir}/python${PYTHON_MAJMIN}/lib-dynload/webbrowser.py"],
>> +#         "rdepends": [],
>> +#         "summary": "Python Web Browser support"}
>> +#
>> +#     * Note that the rdepends field was left empty
>> +#
>> +#     We run $ bitbake python3 -c create_manifest and the resulting manifest
>> +#     should be completed after a few seconds, showing something like:
>> +#     "webbrowser": {
>> +#         "files": ["${libdir}/python${PYTHON_MAJMIN}/webbrowser.py"],
>> +#         "rdepends": ["core","fcntl","io","pickle","shell","subprocess"],
>> +#         "summary": "Python Web Browser support"}
>> +#
>> +#
>> +# If you are upgrading Python to a new version please follow the next steps:
>> +#     After each Python upgrade, the create_manifest task should be executed, because we
>> +#     don't control what changes on upstream Python, so, some module dependency
>> +#     might have changed without us realizing it, a certain module can either have
>> +#     more or less dependencies, or could be depending on a new file that was just
>> +#     created on the new release and for obvious reasons we wouldn't have it on our
>> +#     old manifest, all of these issues would cause runtime errors on our system.
>> +#
>> +#     - Upgrade both the native and target Python packages to a new version
>> +#     - Run the create_manifest task for the target Python package as its shown below:
>> +#
>> +#     $ bitbake python3 -c create_manifest
>> +#
>> +#     This will automatically replace your manifest file located under the Python directory
>> +#     with an new one, which contains the new dependencies (if any).
>> +#
>> +#     Several things could have gone wrong here, I will try to explain a few:
>> +#
>> +#     a) A new file was introduced on this release, e.g. sha3*.so:
>> +#        The task will check what its needed to import every module, more than one module would
>> +#        would probably depend on sha3*.so, although only one module should contain it.
>> +#
>> +#        After running the task, the new manifest will have the sha3*.so file on more than one
>> +#        module, you need to manually decide which one of them should get it and delete it from
>> +#        the others, for example sha3*.so should likely be on ${PN}-crypt.
>> +#        Once you have deleted from the others you need to run the create_manifest task again,
>> +#        this will populate the other module's rdepends fields, with ${PN}-crypt and you should be
>> +#        good to go.
>> +#
>> +#     b) The native package wasn't built correctly and its missing a certain module:
>> +#        As mentioned before, you need to make sure the native package was built with all the modules
>> +#        because it is used as base to build the manifest file, you need to manually check log.do_compile
>> +#        since it won't error out the compile function if its only missing a couple of modules.
>> +#
>> +#        e.g. missing the _uuid module, log.do_compile would show the following:
>> +#        Python build finished successfully!
>> +#        The necessary bits to build these optional modules were not found:
>> +#        _uuid
>> +#
>> +#        What will happen here is that the new manifest would not be aware that the _uuid module exists, so
>> +#        not only we won't know of any dependencies to it, but also, the _uuid* files will be packaged on
>> +#        the misc package (which is where any file that doesn't belong anywhere else ends up).
>> +#
>> +#        This will eventually cause runtime errors on our system if we don't include the misc package on
>> +#        on our image, because the _uuid files will be missing.
>> +#        If we build the _uuid module correctly and run the create_manifest task the _uuid files will be
>> +#        detected correctly along with its dependencies, and we will get a working manifest.
>> +#
>> +#        This is the reason why it is important to make sure we have a fully working native build,
>> +#        so we can avoid these errors.
>> +#
>> +#
>> +#
>> +# DO NOT MODIFY THE NEXT LINE!, IT IS USED AS A MARKER FOR THE ACTUAL JSON MANIFEST
>> +# EOC
>> +{
>> +    "tests": {
>> +        "summary": "Python test suite",
>> +        "rdepends": [
>> +            "core",
>> +            "modules"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/*/test",
>> +            "${libdir}/python${PYTHON_MAJMIN}/*/tests",
>> +            "${libdir}/python${PYTHON_MAJMIN}/idlelib/idle_test/",
>> +            "${libdir}/python${PYTHON_MAJMIN}/test"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "2to3": {
>> +        "summary": "Python automated Python 2 to 3 code translator",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${bindir}/2to3-*",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib2to3"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "asyncio": {
>> +        "summary": "Python Asynchronous I/",
>> +        "rdepends": [
>> +            "core",
>> +            "io",
>> +            "logging",
>> +            "netclient",
>> +            "numbers",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/asyncio",
>> +            "${libdir}/python${PYTHON_MAJMIN}/concurrent",
>> +            "${libdir}/python${PYTHON_MAJMIN}/concurrent/futures",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_asyncio.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "audio": {
>> +        "summary": "Python Audio Handling",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/chunk.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sndhdr.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sunau.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/wave.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc"
>> +        ]
>> +    },
>> +    "codecs": {
>> +        "summary": "Python codec",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/xdrlib.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc"
>> +        ]
>> +    },
>> +    "compile": {
>> +        "summary": "Python bytecode compilation support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/compileall.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/py_compile.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc"
>> +        ]
>> +    },
>> +    "compression": {
>> +        "summary": "Python high-level compression support",
>> +        "rdepends": [
>> +            "core",
>> +            "shell",
>> +            "unixadmin"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/_compression.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/bz2.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/gzip.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bz2.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lzma.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/zlib.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lzma.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/tarfile.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/zipfile.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compression.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bz2.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gzip.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/lzma.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tarfile.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipfile.*.pyc"
>> +        ]
>> +    },
>> +    "core": {
>> +        "summary": "Python interpreter and core modules",
>> +        "rdepends": [],
>> +        "files": [
>> +            "${bindir}/python*[!-config]",
>> +            "${includedir}/python${PYTHON_BINABI}/pyconfig*.h",
>> +            "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/UserList.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/UserString.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__future__.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_abcoll.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_bootlocale.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_markupbase.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_weakrefset.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/abc.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/argparse.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/ast.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/bisect.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/code.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/codecs.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/codeop.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/collections",
>> +            "${libdir}/python${PYTHON_MAJMIN}/collections/abc.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/configparser.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/contextlib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/copy.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/copyreg.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/csv.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/dis.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/aliases.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/latin_1.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/utf_8.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/enum.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/functools.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/genericpath.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/getopt.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/gettext.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/heapq.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/imp.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/_bootstrap_external.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/abc.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/machinery.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/util.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/inspect.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/io.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/keyword.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/_struct.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/binascii.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/time.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/__pycache__/xreadlines.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_bisect.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_csv.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_heapq.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_opcode.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_posixsubprocess.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_struct.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/array.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/binascii.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/math.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/parser.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/readline.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/select.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/time.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/unicodedata.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/xreadlines.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/linecache.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/locale.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/new.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/opcode.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/operator.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/optparse.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/os.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/platform.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/posixpath.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/re.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/reprlib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/rlcompleter.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/selectors.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/signal.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/site.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/stat.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/stringprep.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/struct.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/subprocess.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/symbol.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sysconfig.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/textwrap.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/threading.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/token.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/tokenize.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/traceback.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/types.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/warnings.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/weakref.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/__future__.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_bootlocale.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_collections_abc.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_markupbase.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sitebuiltins.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_sysconfigdata.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_weakrefset.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/abc.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/argparse.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ast.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bisect.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/code.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codecs.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/codeop.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/configparser.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextlib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copy.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/copyreg.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/csv.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/dis.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/enum.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/functools.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/genericpath.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getopt.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/gettext.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/heapq.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imp.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/inspect.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/io.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/keyword.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/linecache.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/locale.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/opcode.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/operator.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/optparse.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/os.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/platform.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/posixpath.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/re.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/reprlib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/rlcompleter.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/selectors.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/signal.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/site.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_compile.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_constants.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sre_parse.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stat.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/stringprep.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/struct.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/subprocess.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/symbol.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/sysconfig.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/textwrap.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/threading.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/token.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tokenize.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/traceback.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/types.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/warnings.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/weakref.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__",
>> +            "${libdir}/python${PYTHON_MAJMIN}/collections/__pycache__/abc.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/aliases.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/latin_1.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/encodings/__pycache__/utf_8.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/abc.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/machinery.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/importlib/__pycache__/util.*.pyc"
>> +        ]
>> +    },
>> +    "crypt": {
>> +        "summary": "Python basic cryptographic and hashing support",
>> +        "rdepends": [
>> +            "core",
>> +            "math",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/crypt.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/hashlib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_crypt.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_hashlib.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha256.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha512.*.so"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/crypt.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hashlib.*.pyc"
>> +        ]
>> +    },
>> +    "ctypes": {
>> +        "summary": "Python C types support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/ctypes",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ctypes_test.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "curses": {
>> +        "summary": "Python curses support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/curses",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_curses_panel.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "datetime": {
>> +        "summary": "Python calendar and time support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/_strptime.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/calendar.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/datetime.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_datetime.*.so"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_strptime.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/calendar.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/datetime.*.pyc"
>> +        ]
>> +    },
>> +    "db": {
>> +        "summary": "Python file-based database support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/dbm",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_dbm.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "debugger": {
>> +        "summary": "Python debugger",
>> +        "rdepends": [
>> +            "core",
>> +            "pprint",
>> +            "shell",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/bdb.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pdb.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/bdb.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pdb.*.pyc"
>> +        ]
>> +    },
>> +    "dev": {
>> +        "cached": [],
>> +        "files": [
>> +            "${base_libdir}/*.a",
>> +            "${base_libdir}/*.o",
>> +            "${bindir}/python*-config",
>> +            "${datadir}/aclocal",
>> +            "${datadir}/pkgconfig",
>> +            "${includedir}",
>> +            "${libdir}/*.a",
>> +            "${libdir}/*.la",
>> +            "${libdir}/*.o",
>> +            "${libdir}/lib*${SOLIBSDEV}",
>> +            "${libdir}/pkgconfig",
>> +            "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile",
>> +            "${libdir}/python${PYTHON_MAJMIN}/config*/Makefile/__pycache__"
>> +        ],
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "summary": "Python development package"
>> +    },
>> +    "difflib": {
>> +        "summary": "Python helpers for computing deltas between objects",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/difflib.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/difflib.*.pyc"
>> +        ]
>> +    },
>> +    "distutils-staticdev": {
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/config/__pycache__/lib*.a"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/config/lib*.a"
>> +        ],
>> +        "rdepends": [
>> +            "distutils"
>> +        ],
>> +        "summary": "Python distribution utilities (static libraries)"
>> +    },
>> +    "distutils": {
>> +        "summary": "Python Distribution Utilities",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/distutils"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "doctest": {
>> +        "summary": "Python framework for running examples in docstrings",
>> +        "rdepends": [
>> +            "core",
>> +            "debugger",
>> +            "difflib",
>> +            "logging",
>> +            "pprint",
>> +            "shell",
>> +            "stringold",
>> +            "unittest"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/doctest.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/doctest.*.pyc"
>> +        ]
>> +    },
>> +    "email": {
>> +        "summary": "Python email support",
>> +        "rdepends": [
>> +            "core",
>> +            "crypt",
>> +            "datetime",
>> +            "io",
>> +            "math",
>> +            "netclient"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/email",
>> +            "${libdir}/python${PYTHON_MAJMIN}/imaplib.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imaplib.*.pyc"
>> +        ]
>> +    },
>> +    "fcntl": {
>> +        "summary": "Python's fcntl interface",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/fcntl.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "gdbm": {
>> +        "summary": "Python GNU database support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_gdbm.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "html": {
>> +        "summary": "Python HTML processing support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/formatter.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/html"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/formatter.*.pyc"
>> +        ]
>> +    },
>> +    "idle": {
>> +        "summary": "Python Integrated Development Environment",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${bindir}/idle*",
>> +            "${libdir}/python${PYTHON_MAJMIN}/idlelib"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "image": {
>> +        "summary": "Python graphical image handling",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/colorsys.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/imghdr.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/colorsys.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/imghdr.*.pyc"
>> +        ]
>> +    },
>> +    "io": {
>> +        "summary": "Python low-level I/O",
>> +        "rdepends": [
>> +            "compression",
>> +            "core",
>> +            "crypt",
>> +            "math",
>> +            "netclient",
>> +            "shell",
>> +            "unixadmin"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/_pyio.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/ipaddress.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_socket.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_ssl.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/termios.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pipes.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/socket.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/ssl.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/tempfile.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pyio.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ipaddress.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pipes.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socket.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ssl.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tempfile.*.pyc"
>> +        ]
>> +    },
>> +    "json": {
>> +        "summary": "Python JSON support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/json",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_json.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "logging": {
>> +        "summary": "Python logging support",
>> +        "rdepends": [
>> +            "core",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/logging"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "mailbox": {
>> +        "summary": "Python mailbox format support",
>> +        "rdepends": [
>> +            "core",
>> +            "crypt",
>> +            "datetime",
>> +            "email",
>> +            "fcntl",
>> +            "io",
>> +            "math",
>> +            "mime",
>> +            "netclient",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/mailbox.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mailbox.*.pyc"
>> +        ]
>> +    },
>> +    "math": {
>> +        "summary": "Python math support",
>> +        "rdepends": [
>> +            "core",
>> +            "crypt"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_random.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/cmath.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/random.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/random.*.pyc"
>> +        ]
>> +    },
>> +    "mime": {
>> +        "summary": "Python MIME handling APIs",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/quopri.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/uu.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/quopri.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uu.*.pyc"
>> +        ]
>> +    },
>> +    "mmap": {
>> +        "summary": "Python memory-mapped file support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/mmap.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "modules": {
>> +        "cached": [],
>> +        "files": [],
>> +        "rdepends": [
>> +            "2to3",
>> +            "asyncio",
>> +            "audio",
>> +            "codecs",
>> +            "compile",
>> +            "compression",
>> +            "core",
>> +            "crypt",
>> +            "ctypes",
>> +            "curses",
>> +            "datetime",
>> +            "db",
>> +            "debugger",
>> +            "difflib",
>> +            "distutils",
>> +            "doctest",
>> +            "email",
>> +            "fcntl",
>> +            "html",
>> +            "idle",
>> +            "image",
>> +            "io",
>> +            "json",
>> +            "logging",
>> +            "mailbox",
>> +            "math",
>> +            "mime",
>> +            "mmap",
>> +            "multiprocessing",
>> +            "netclient",
>> +            "netserver",
>> +            "numbers",
>> +            "pickle",
>> +            "pkgutil",
>> +            "plistlib",
>> +            "pprint",
>> +            "profile",
>> +            "pydoc",
>> +            "resource",
>> +            "runpy",
>> +            "shell",
>> +            "smtpd",
>> +            "sqlite3",
>> +            "stringold",
>> +            "syslog",
>> +            "terminal",
>> +            "threading",
>> +            "tkinter",
>> +            "typing",
>> +            "unittest",
>> +            "unixadmin",
>> +            "venv",
>> +            "xml",
>> +            "xmlrpc"
>> +        ],
>> +        "summary": "All Python modules"
>> +    },
>> +    "multiprocessing": {
>> +        "summary": "Python multiprocessing support",
>> +        "rdepends": [
>> +            "core",
>> +            "io",
>> +            "pickle"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multiprocessing.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/multiprocessing"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "netclient": {
>> +        "summary": "Python Internet Protocol clients",
>> +        "rdepends": [
>> +            "core",
>> +            "crypt",
>> +            "datetime",
>> +            "email",
>> +            "io",
>> +            "math",
>> +            "mime",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/base64.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/ftplib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/hmac.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/http",
>> +            "${libdir}/python${PYTHON_MAJMIN}/http/__pycache__",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_uuid.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/mimetypes.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/nntplib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/poplib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/smtplib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/telnetlib.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/urllib",
>> +            "${libdir}/python${PYTHON_MAJMIN}/urllib/__pycache__",
>> +            "${libdir}/python${PYTHON_MAJMIN}/uuid.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/base64.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/ftplib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/hmac.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/mimetypes.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/nntplib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/poplib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtplib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/telnetlib.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/uuid.*.pyc"
>> +        ]
>> +    },
>> +    "netserver": {
>> +        "summary": "Python Internet Protocol servers",
>> +        "rdepends": [
>> +            "compression",
>> +            "core",
>> +            "crypt",
>> +            "datetime",
>> +            "email",
>> +            "html",
>> +            "io",
>> +            "math",
>> +            "mime",
>> +            "netclient",
>> +            "shell",
>> +            "stringold",
>> +            "unixadmin"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/cgi.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/socketserver.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgi.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/socketserver.*.pyc"
>> +        ]
>> +    },
>> +    "numbers": {
>> +        "summary": "Python number APIs",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/_pydecimal.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/contextvars.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/decimal.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/fractions.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_decimal.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/numbers.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_pydecimal.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/decimal.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fractions.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/numbers.*.pyc"
>> +        ]
>> +    },
>> +    "pickle": {
>> +        "summary": "Python serialisation/persistence support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/_compat_pickle.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_pickle.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pickle.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pickletools.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/shelve.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_compat_pickle.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickle.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pickletools.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shelve.*.pyc"
>> +        ]
>> +    },
>> +    "pkgutil": {
>> +        "summary": "Python package extension utility support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/pkgutil.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pkgutil.*.pyc"
>> +        ]
>> +    },
>> +    "plistlib": {
>> +        "summary": "Generate and parse Mac OS X .plist files",
>> +        "rdepends": [
>> +            "core",
>> +            "datetime",
>> +            "xml"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/plistlib.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/plistlib.*.pyc"
>> +        ]
>> +    },
>> +    "pprint": {
>> +        "summary": "Python pretty-print support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/pprint.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pprint.*.pyc"
>> +        ]
>> +    },
>> +    "profile": {
>> +        "summary": "Python basic performance profiling support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/cProfile.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_lsprof.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/profile.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pstats.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cProfile.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/profile.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pstats.*.pyc"
>> +        ]
>> +    },
>> +    "pydoc": {
>> +        "summary": "Python interactive help support",
>> +        "rdepends": [
>> +            "core",
>> +            "netclient",
>> +            "pkgutil"
>> +        ],
>> +        "files": [
>> +            "${bindir}/pydoc*",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pydoc.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/pydoc_data"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pydoc.*.pyc"
>> +        ]
>> +    },
>> +    "resource": {
>> +        "summary": "Python resource control interface",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/resource.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "runpy": {
>> +        "summary": "Python helper for locating/executing scripts in module namespace",
>> +        "rdepends": [
>> +            "core",
>> +            "pkgutil"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/runpy.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/runpy.*.pyc"
>> +        ]
>> +    },
>> +    "shell": {
>> +        "summary": "Python shell-like functionality",
>> +        "rdepends": [
>> +            "compression",
>> +            "core",
>> +            "stringold",
>> +            "unixadmin"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/cmd.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/fnmatch.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/glob.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/shlex.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/shutil.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cmd.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/fnmatch.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/glob.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shlex.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/shutil.*.pyc"
>> +        ]
>> +    },
>> +    "smtpd": {
>> +        "summary": "Python Simple Mail Transport Daemon",
>> +        "rdepends": [
>> +            "core",
>> +            "crypt",
>> +            "datetime",
>> +            "email",
>> +            "io",
>> +            "math",
>> +            "mime",
>> +            "netclient",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${bindir}/smtpd.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/asynchat.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/asyncore.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/smtpd.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asynchat.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/asyncore.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/smtpd.*.pyc"
>> +        ]
>> +    },
>> +    "sqlite3": {
>> +        "summary": "Python Sqlite3 database support",
>> +        "rdepends": [
>> +            "core",
>> +            "datetime"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sqlite3.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/sqlite3"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "stringold": {
>> +        "summary": "Python string APIs [deprecated]",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/string.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/string.*.pyc"
>> +        ]
>> +    },
>> +    "syslog": {
>> +        "summary": "Python syslog interface",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/syslog.*.so"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "terminal": {
>> +        "summary": "Python terminal controlling support",
>> +        "rdepends": [
>> +            "core",
>> +            "io"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/pty.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/tty.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/pty.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/tty.*.pyc"
>> +        ]
>> +    },
>> +    "threading": {
>> +        "summary": "Python threading & synchronization support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/_dummy_thread.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/_threading_local.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/queue.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_dummy_thread.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/_threading_local.*.pyc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/queue.*.pyc"
>> +        ]
>> +    },
>> +    "tkinter": {
>> +        "summary": "Python Tcl/Tk bindings",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_tkinter.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/tkinter"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "typing": {
>> +        "summary": "Python typing support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/typing.py"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/typing.*.pyc"
>> +        ]
>> +    },
>> +    "unittest": {
>> +        "summary": "Python unit testing framework",
>> +        "rdepends": [
>> +            "core",
>> +            "difflib",
>> +            "logging",
>> +            "pprint",
>> +            "shell",
>> +            "stringold"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/unittest",
>> +            "${libdir}/python${PYTHON_MAJMIN}/unittest/",
>> +            "${libdir}/python${PYTHON_MAJMIN}/unittest/__pycache__"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "unixadmin": {
>> +        "summary": "Python Unix administration support",
>> +        "rdepends": [
>> +            "core",
>> +            "io"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/getpass.py",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/grp.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/nis.*.so"
>> +        ],
>> +        "cached": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/getpass.*.pyc"
>> +        ]
>> +    },
>> +    "venv": {
>> +        "summary": "Provides support for creating lightweight virtual environments with their own site directories, optionally isolated from system site directories.",
>> +        "rdepends": [
>> +            "compression",
>> +            "core",
>> +            "logging",
>> +            "shell",
>> +            "stringold",
>> +            "unixadmin"
>> +        ],
>> +        "files": [
>> +            "${bindir}/pyvenv*",
>> +            "${libdir}/python${PYTHON_MAJMIN}/venv"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "xml": {
>> +        "summary": "Python basic XML support",
>> +        "rdepends": [
>> +            "core"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_elementtree.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/lib-dynload/pyexpat.*.so",
>> +            "${libdir}/python${PYTHON_MAJMIN}/xml"
>> +        ],
>> +        "cached": []
>> +    },
>> +    "xmlrpc": {
>> +        "summary": "Python XML-RPC support",
>> +        "rdepends": [
>> +            "core",
>> +            "xml"
>> +        ],
>> +        "files": [
>> +            "${libdir}/python${PYTHON_MAJMIN}/xmlrpc",
>> +            "${libdir}/python${PYTHON_MAJMIN}/xmlrpc/__pycache__"
>> +        ],
>> +        "cached": []
>> +    }
>> +}
>> diff --git a/meta/recipes-devtools/python-sanity/python3/run-ptest b/meta/recipes-devtools/python-sanity/python3/run-ptest
>> new file mode 100644
>> index 00000000000..3863c6d314f
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3/run-ptest
>> @@ -0,0 +1,3 @@
>> +#!/bin/sh
>> +
>> +python3 -m test -v | sed -e '/\.\.\. ok/ s/^/PASS: /g' -e '/\.\.\. [ERROR|FAIL]/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g'
>> diff --git a/meta/recipes-devtools/python-sanity/python3_3.7.2.bb b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
>> new file mode 100644
>> index 00000000000..61fa12ffe8d
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python-sanity/python3_3.7.2.bb
>> @@ -0,0 +1,281 @@
>> +SUMMARY = "The Python Programming Language"
>> +HOMEPAGE = "http://www.python.org"
>> +LICENSE = "PSFv2"
>> +SECTION = "devel/python"
>> +
>> +LIC_FILES_CHKSUM = "file://LICENSE;md5=f257cc14f81685691652a3d3e1b5d754"
>> +
>> +SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
>> +           file://run-ptest \
>> +           file://create_manifest3.py \
>> +           file://get_module_deps3.py \
>> +           file://python3-manifest.json \
>> +           file://check_build_completeness.py \
>> +           file://cgi_py.patch \
>> +           file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \
>> +           ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \
>> +           file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \
>> +           file://python-config.patch \
>> +           file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \
>> +           "
>> +
>> +SRC_URI_append_class-native = " \
>> +           file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
>> +           file://12-distutils-prefix-is-inside-staging-area.patch \
>> +           "
>> +
>> +SRC_URI[md5sum] = "df6ec36011808205beda239c72f947cb"
>> +SRC_URI[sha256sum] = "d83fe8ce51b1bb48bbcf0550fd265b9a75cdfdfa93f916f9e700aef8444bf1bb"
>> +
>> +# exclude pre-releases for both python 2.x and 3.x
>> +UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P<pver>\d+(\.\d+)+).tar"
>> +
>> +CVE_PRODUCT = "python"
>> +
>> +PYTHON_MAJMIN = "3.7"
>> +PYTHON_BINABI = "${PYTHON_MAJMIN}m"
>> +
>> +S = "${WORKDIR}/Python-${PV}"
>> +
>> +BBCLASSEXTEND = "native nativesdk"
>> +
>> +inherit autotools pkgconfig qemu ptest multilib_header update-alternatives
>> +
>> +MULTILIB_SUFFIX = "${@d.getVar('base_libdir',1).split('/')[-1]}"
>> +
>> +ALTERNATIVE_${PN}-dev = "python-config"
>> +ALTERNATIVE_LINK_NAME[python-config] = "${bindir}/python${PYTHON_BINABI}-config"
>> +ALTERNATIVE_TARGET[python-config] = "${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}"
>> +
>> +
>> +DEPENDS = "bzip2-replacement-native libffi bzip2 gdbm openssl sqlite3 zlib virtual/libintl xz virtual/crypt util-linux libtirpc libnsl2"
>> +DEPENDS_append_class-target = " python3-native"
>> +DEPENDS_append_class-nativesdk = " python3-native"
>> +
>> +EXTRA_OECONF = " --without-ensurepip --enable-shared"
>> +EXTRA_OECONF_append_class-native = " --bindir=${bindir}/${PN}"
>> +
>> +
>> +EXTRANATIVEPATH += "python3-native"
>> +
>> +CACHED_CONFIGUREVARS = " \
>> +                ac_cv_file__dev_ptmx=yes \
>> +                ac_cv_file__dev_ptc=no \
>> +"
>> +
>> +PACKAGECONFIG_class-target ??= "readline ${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'pgo', '', d)}"
>> +PACKAGECONFIG_class-native ??= "readline"
>> +PACKAGECONFIG_class-nativesdk ??= "readline"
>> +PACKAGECONFIG[readline] = ",,readline"
>> +# Use profile guided optimisation by running PyBench inside qemu-user
>> +PACKAGECONFIG[pgo] = "--enable-optimizations,,qemu-helper-native"
>> +PACKAGECONFIG[tk] = ",,tk"
>> +
>> +CPPFLAGS_append = " -I${STAGING_INCDIR}/ncursesw -I${STAGING_INCDIR}/uuid"
>> +
>> +EXTRA_OEMAKE = '\
>> +  STAGING_LIBDIR=${STAGING_LIBDIR} \
>> +  STAGING_INCDIR=${STAGING_INCDIR} \
>> +'
>> +
>> +do_compile_prepend_class-target() {
>> +       if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then
>> +                qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}"
>> +                cat >pgo-wrapper <<EOF
>> +#!/bin/sh
>> +cd ${B}
>> +$qemu_binary "\$@"
>> +EOF
>> +                chmod +x pgo-wrapper
>> +        fi
>> +}
>> +
>> +do_install_prepend() {
>> +        ${WORKDIR}/check_build_completeness.py ${T}/log.do_compile
>> +}
>> +
>> +do_install_append_class-target() {
>> +        oe_multilib_header python${PYTHON_BINABI}/pyconfig.h
>> +}
>> +
>> +do_install_append_class-native() {
>> +        # Make sure we use /usr/bin/env python
>> +        for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do
>> +                sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
>> +        done
>> +        # Add a symlink to the native Python so that scripts can just invoke
>> +        # "nativepython" and get the right one without needing absolute paths
>> +        # (these often end up too long for the #! parser in the kernel as the
>> +        # buffer is 128 bytes long).
>> +        ln -s python3-native/python3 ${D}${bindir}/nativepython3
>> +}
>> +
>> +do_install_append() {
>> +        mkdir -p ${D}${libdir}/python-sysconfigdata
>> +        sysconfigfile=`find ${D} -name _sysconfig*.py`
>> +        cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py
>> +
>> +        sed -i  \
>> +                -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \
>> +                -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
>> +                -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
>> +                ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py
>> +}
>> +
>> +SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py"
>> +PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
>> +
>> +py_package_preprocess () {
>> +        # Remove references to buildmachine paths in target Makefile and _sysconfigdata
>> +        sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \
>> +                -e 's|${DEBUG_PREFIX_MAP}||g' \
>> +                -e 's:${HOSTTOOLS_DIR}/::g' \
>> +                -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
>> +                -e 's:${RECIPE_SYSROOT}::g' \
>> +                -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
>> +                ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_MAJMIN}${PYTHON_ABI}*/Makefile \
>> +                ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata*.py \
>> +                ${PKGD}/${bindir}/python${PYTHON_BINABI}-config
>> +
>> +        # Recompile _sysconfigdata after modifying it
>> +        cd ${PKGD}
>> +        sysconfigfile=`find . -name _sysconfigdata_*.py`
>> +        ${STAGING_BINDIR_NATIVE}/python3-native/python3 \
>> +             -c "from py_compile import compile; compile('$sysconfigfile')"
>> +        ${STAGING_BINDIR_NATIVE}/python3-native/python3 \
>> +             -c "from py_compile import compile; compile('$sysconfigfile', optimize=1)"
>> +        ${STAGING_BINDIR_NATIVE}/python3-native/python3 \
>> +             -c "from py_compile import compile; compile('$sysconfigfile', optimize=2)"
>> +        cd -
>> +
>> +        mv ${PKGD}/${bindir}/python${PYTHON_BINABI}-config ${PKGD}/${bindir}/python${PYTHON_BINABI}-config-${MULTILIB_SUFFIX}
>> +
>> +        #Remove the unneeded copy of target sysconfig data
>> +        rm -rf ${PKGD}/${libdir}/python-sysconfigdata
>> +}
>> +
>> +# We want bytecode precompiled .py files (.pyc's) by default
>> +# but the user may set it on their own conf
>> +INCLUDE_PYCS ?= "1"
>> +
>> +python(){
>> +    import collections, json
>> +
>> +    filename = os.path.join(d.getVar('THISDIR'), 'python3', 'python3-manifest.json')
>> +    # This python changes the datastore based on the contents of a file, so mark
>> +    # that dependency.
>> +    bb.parse.mark_dependency(d, filename)
>> +
>> +    with open(filename) as manifest_file:
>> +        manifest_str =  manifest_file.read()
>> +        json_start = manifest_str.find('# EOC') + 6
>> +        manifest_file.seek(json_start)
>> +        manifest_str = manifest_file.read()
>> +        python_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict)
>> +
>> +    # First set RPROVIDES for -native case
>> +    # Hardcoded since it cant be python3-native-foo, should be python3-foo-native
>> +    pn = 'python3'
>> +    rprovides = d.getVar('RPROVIDES').split()
>> +
>> +    for key in python_manifest:
>> +        pypackage = pn + '-' + key + '-native'
>> +        if pypackage not in rprovides:
>> +              rprovides.append(pypackage)
>> +
>> +    d.setVar('RPROVIDES_class-native', ' '.join(rprovides))
>> +
>> +    # Then work on the target
>> +    include_pycs = d.getVar('INCLUDE_PYCS')
>> +
>> +    packages = d.getVar('PACKAGES').split()
>> +    pn = d.getVar('PN')
>> +
>> +    newpackages=[]
>> +    for key in python_manifest:
>> +        pypackage= pn + '-' + key
>> +
>> +        if pypackage not in packages:
>> +            # We need to prepend, otherwise python-misc gets everything
>> +            # so we use a new variable
>> +            newpackages.append(pypackage)
>> +
>> +        # "Build" python's manifest FILES, RDEPENDS and SUMMARY
>> +        d.setVar('FILES_' + pypackage, '')
>> +        for value in python_manifest[key]['files']:
>> +            d.appendVar('FILES_' + pypackage, ' ' + value)
>> +
>> +        # Add cached files
>> +        if include_pycs == '1':
>> +            for value in python_manifest[key]['cached']:
>> +                    d.appendVar('FILES_' + pypackage, ' ' + value)
>> +
>> +        for value in python_manifest[key]['rdepends']:
>> +            # Make it work with or without $PN
>> +            if '${PN}' in value:
>> +                value=value.split('-')[1]
>> +            d.appendVar('RDEPENDS_' + pypackage, ' ' + pn + '-' + value)
>> +        d.setVar('SUMMARY_' + pypackage, python_manifest[key]['summary'])
>> +
>> +    # Prepending so to avoid python-misc getting everything
>> +    packages = newpackages + packages
>> +    d.setVar('PACKAGES', ' '.join(packages))
>> +    d.setVar('ALLOW_EMPTY_${PN}-modules', '1')
>> +}
>> +
>> +# Files needed to create a new manifest
>> +
>> +do_create_manifest() {
>> +    # This task should be run with every new release of Python.
>> +    # We must ensure that PACKAGECONFIG enables everything when creating
>> +    # a new manifest, this is to base our new manifest on a complete
>> +    # native python build, containing all dependencies, otherwise the task
>> +    # wont be able to find the required files.
>> +    # e.g. BerkeleyDB is an optional build dependency so it may or may not
>> +    # be present, we must ensure it is.
>> +
>> +    cd ${WORKDIR}
>> +    # This needs to be executed by python-native and NOT by HOST's python
>> +    nativepython3 create_manifest3.py ${PYTHON_MAJMIN}
>> +    cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json
>> +}
>> +
>> +# bitbake python -c create_manifest
>> +addtask do_create_manifest
>> +
>> +# Make sure we have native python ready when we create a new manifest
>> +do_create_manifest[depends] += "python3:do_prepare_recipe_sysroot"
>> +do_create_manifest[depends] += "python3:do_patch"
>> +
>> +# manual dependency additions
>> +RPROVIDES_${PN}-modules = "${PN}"
>> +RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules"
>> +RRECOMMENDS_${PN}-crypt = "openssl ca-certificates"
>> +
>> +FILES_${PN}-2to3 += "${bindir}/2to3-${PYTHON_MAJMIN}"
>> +FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
>> +FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}"
>> +
>> +# provide python-pyvenv from python3-venv
>> +RPROVIDES_${PN}-venv += "python3-pyvenv"
>> +
>> +# package libpython3
>> +PACKAGES =+ "libpython3 libpython3-staticdev"
>> +FILES_libpython3 = "${libdir}/libpython*.so.*"
>> +FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}-*/libpython${PYTHON_BINABI}.a"
>> +INSANE_SKIP_${PN}-dev += "dev-elf"
>> +
>> +# catch all the rest (unsorted)
>> +PACKAGES += "${PN}-misc"
>> +RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs"
>> +RDEPENDS_${PN}-modules += "python3-misc"
>> +FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN}"
>> +
>> +# catch manpage
>> +PACKAGES += "${PN}-man"
>> +FILES_${PN}-man = "${datadir}/man"
>> +
>> +RDEPENDS_${PN}-ptest = "${PN}-modules ${PN}-tests unzip bzip2"
>> +RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk', '', d)}"
>> +RDEPENDS_${PN}-dev = ""
>> +
>> --
>> 2.17.1
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190201/8dbbbd9d/attachment-0001.html>


More information about the Openembedded-core mailing list