[OE-core] [PATCH] gpg_sign: perform rpm signing serially

[Mark Hatle]

It would probably be better if this was configurable with a 'safe' default.

Moving from parallel to single will greatly affect the overall performance on
larger build machines (lots of memory and cores) that can handle the load vs a
typical development machine.

--Mark

On 8/15/17 4:40 PM, leonardo.sandoval.gonzalez at linux.intel.com wrote:
> From: Leonardo Sandoval <leonardo.sandoval.gonzalez at linux.intel.com>
> 
> gpg signing in file batches (which was default to 100) is a memory expensive
> computation, causing trouble in some host machines (even on production AB
> as seen on the bugzilla ID). Also, in terms of performance, there is no real
> gain when rpm signing is done in batches. Considering the latter issues, perform the
> rpm signing serially.
> 
> Log showing errors observed recently at AB workers:
> 
>     | gpg: signing failed: Cannot allocate memory
>     | gpg: signing failed: Cannot allocate memory
>     | error: gpg exec failed (2)
>     | /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/core2-64-poky-linux/base-passwd/3.5.29-r0/deploy-rpms/core2_64/base-passwd-dev-3.5.29-r0.core2_64.rpm:
> 
> [YOCTO #11914]
> 
> Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez at linux.intel.com>
> ---
>  meta/lib/oe/gpg_sign.py | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/lib/oe/gpg_sign.py b/meta/lib/oe/gpg_sign.py
> index f4d8b10e4b..5c7985a856 100644
> --- a/meta/lib/oe/gpg_sign.py
> +++ b/meta/lib/oe/gpg_sign.py
> @@ -45,9 +45,9 @@ class LocalSigner(object):
>              if fsk_password:
>                  cmd += "--define '_file_signing_key_password %s' " % fsk_password
>  
> -        # Sign in chunks of 100 packages
> -        for i in range(0, len(files), 100):
> -            status, output = oe.utils.getstatusoutput(cmd + ' '.join(files[i:i+100]))
> +        # Sign packages
> +        for f in files:
> +            status, output = oe.utils.getstatusoutput(cmd + ' ' + f)
>              if status:
>                  raise bb.build.FuncFailed("Failed to sign RPM packages: %s" % output)
>  
>