[bitbake-devel] [PATCH master 1.42] fetch2: runfetchcmd(): unset _PYTHON_SYSCONFIGDATA_NAME

[Matthias Schiffer]

On Wed, 2019-05-22 at 11:13 +0200, Matthias Schiffer wrote:
> Since warrior, python3native.bbclass sets _PYTHON_SYSCONFIGDATA_NAME;
> unfortunately, this also affects Python scripts run as fetch commands
> like
> git-make-shallow, breaking it with a message like
> 
>     Failed to import the site module
>     Traceback (most recent call last):
>       File "/usr/lib/python3.6/site.py", line 570, in <module>
>         main()
>       File "/usr/lib/python3.6/site.py", line 556, in main
>         known_paths = addusersitepackages(known_paths)
>       File "/usr/lib/python3.6/site.py", line 288, in
> addusersitepackages
>         user_site = getusersitepackages()
>       File "/usr/lib/python3.6/site.py", line 264, in
> getusersitepackages
>         user_base = getuserbase() # this will also set USER_BASE
>       File "/usr/lib/python3.6/site.py", line 254, in getuserbase
>         USER_BASE = get_config_var('userbase')
>       File "/usr/lib/python3.6/sysconfig.py", line 607, in
> get_config_var
>         return get_config_vars().get(name)
>       File "/usr/lib/python3.6/sysconfig.py", line 550, in
> get_config_vars
>         _init_posix(_CONFIG_VARS)
>       File "/usr/lib/python3.6/sysconfig.py", line 421, in
> _init_posix
>         _temp = __import__(name, globals(), locals(),
> ['build_time_vars'], 0)
>     ModuleNotFoundError: No module named '_sysconfigdata'
> 
> on an Ubuntu 18.04 system (and likely others) when building with
> BB_GIT_SHALLOW and BB_GENERATE_SHALLOW_TARBALLS.
> 
> Unset _PYTHON_SYSCONFIGDATA_NAME in runfetchcmd() to work around
> this.
> 
> Signed-off-by: Matthias Schiffer <matthias.schiffer at ew.tq-group.com>
> ---
> 
> I think this (or a different fix for this issue) should be added to
> both
> master and 1.42, as Yocto warrior is affected.

Thanks, this has been in master for ~3 weeks. Please also apply to
1.42, as we're currently seeing this issue with Yocto warrior.

Regards,
Matthias


> 
> Note that this fix assumes that the host Python does not need a
> _PYTHON_SYSCONFIGDATA_NAME, which should be true on most systems. A
> more
> complete fix would be to reset _PYTHON_SYSCONFIGDATA_NAME to its
> original
> value, but I don't know the Bitbake code well enough to do that.
> 
>  lib/bb/fetch2/__init__.py | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
> index 0126e0d7..f6b5529b 100644
> --- a/lib/bb/fetch2/__init__.py
> +++ b/lib/bb/fetch2/__init__.py
> @@ -843,6 +843,11 @@ def runfetchcmd(cmd, d, quiet=False,
> cleanup=None, log=None, workdir=None):
>          if val:
>              cmd = 'export ' + var + '=\"%s\"; %s' % (val, cmd)
>  
> +    # Ensure that a _PYTHON_SYSCONFIGDATA_NAME value set by a recipe
> +    # (for example via python3native.bbclass since warrior) is not
> set for
> +    # host Python (otherwise tools like git-make-shallow will fail)
> +    cmd = 'unset _PYTHON_SYSCONFIGDATA_NAME; ' + cmd
> +
>      # Disable pseudo as it may affect ssh, potentially causing it to
> hang.
>      cmd = 'export PSEUDO_DISABLED=1; ' + cmd
>