[OE-core] [PATCH] yocto-compat-layer: improve error handling in signature creation

[Aníbal Limón]

Acked-by:  Aníbal Limón <anibal.limon at linux.intel.com>

On 03/15/2017 04:01 AM, Patrick Ohly wrote:
> When "bitbake -k -S none world" failed, the error printed by
> yocto-compat-layer.py contained the stack trace multiple times and did not
> contain the stderr output from bitbake, making the error hard to understand
> and debug:
> 
>   INFO: ======================================================================
>   INFO: ERROR: test_signatures (common.CommonCompatLayer)
>   INFO: ----------------------------------------------------------------------
>   INFO: Traceback (most recent call last):
>     File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
>       stderr=subprocess.PIPE)
>     File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
>       raise CalledProcessError(retcode, process.args, output=output)
>   subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
> 
>   During handling of the above exception, another exception occurred:
> 
>   Traceback (most recent call last):
>     File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
>       curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
>     File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in get_signatures
>       raise RuntimeError(msg)
>   RuntimeError: Traceback (most recent call last):
>     File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures
>       stderr=subprocess.PIPE)
>     File "/usr/lib/python3.4/subprocess.py", line 620, in check_output
>       raise CalledProcessError(retcode, process.args, output=output)
>   subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1
> 
>   Loading cache...done.
>   Loaded 1328 entries from dependency cache.
>   NOTE: Resolving any missing task queue dependencies
>   NOTE: Runtime target 'zlib-qat' is unbuildable, removing...
>   Missing or unbuildable dependency chain was: ['zlib-qat']
>   ...
>   Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
> 
> The yocto-compat-layer.log was incomplete, it only had the first part
> without the command output.
> 
> stderr was missing due to stderr=subprocess.PIPE.
> 
> Instead of the complicated try/except construct it's better to check
> the return code ourselves and raise just a single exception. The
> output (both on stderr and in the yocto-compat-layer.log) now is:
> 
>   INFO: ======================================================================
>   INFO: ERROR: test_signatures (common.CommonCompatLayer)
>   INFO: ----------------------------------------------------------------------
>   INFO: Traceback (most recent call last):
>     File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures
>       curr_sigs = get_signatures(self.td['builddir'], failsafe=True)
>     File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in get_signatures
>       raise RuntimeError(msg)
>   RuntimeError: Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.
>   Command: bitbake -k -S none world
>   Output:
>   Loading cache...done.
>   Loaded 1328 entries from dependency cache.
>   NOTE: Resolving any missing task queue dependencies
>   ERROR: Nothing PROVIDES 'qat16' (but /fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb DEPENDS on or otherwise requires it)
>   ERROR: qat16 was skipped: incompatible with machine qemux86 (not in COMPATIBLE_MACHINE)
>   ...
>   Missing or unbuildable dependency chain was: ['openssl-qat-dev']
>   ...
>   Summary: There were 5 ERROR messages shown, returning a non-zero exit code.
> 
> Signed-off-by: Patrick Ohly <patrick.ohly at intel.com>
> ---
>  scripts/lib/compatlayer/__init__.py | 20 +++++++++-----------
>  1 file changed, 9 insertions(+), 11 deletions(-)
> 
> diff --git a/scripts/lib/compatlayer/__init__.py b/scripts/lib/compatlayer/__init__.py
> index b3a166aa9ab..a7eb8625310 100644
> --- a/scripts/lib/compatlayer/__init__.py
> +++ b/scripts/lib/compatlayer/__init__.py
> @@ -135,17 +135,15 @@ def get_signatures(builddir, failsafe=False):
>  
>      sigs = {}
>  
> -    try:
> -        cmd = 'bitbake '
> -        if failsafe:
> -            cmd += '-k '
> -        cmd += '-S none world'
> -        output = subprocess.check_output(cmd, shell=True,
> -                stderr=subprocess.PIPE)
> -    except subprocess.CalledProcessError as e:
> -        import traceback
> -        exc = traceback.format_exc()
> -        msg = '%s\n%s\n' % (exc, e.output.decode('utf-8'))
> +    cmd = 'bitbake '
> +    if failsafe:
> +        cmd += '-k '
> +    cmd += '-S none world'
> +    p = subprocess.Popen(cmd, shell=True,
> +                         stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
> +    output, _ = p.communicate()
> +    if p.returncode:
> +        msg = "Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.\nCommand: %s\nOutput:\n%s" % (cmd, output.decode('utf-8'))
>          raise RuntimeError(msg)
>      sigs_file = os.path.join(builddir, 'locked-sigs.inc')
>  
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20170315/dfb13198/attachment-0002.sig>