[bitbake-devel] [PATCH 2/7] data_smart: Micro optimise _remove handling

Fri Oct 19 08:57:38 UTC 2018

This one and "data_smart: Fix expand_cache and _remove operator interaction
issues" are slightly changing the behavior, not sure if this change was
intended.

Since last bitbake upgrade I've started seeing:
ERROR: ca-certificates-20180409-r0 do_package_qa: QA Issue: ca-certificates
rdepends on openssl, but it isn't a build dependency? [build-deps]
ERROR: nativesdk-bash-completion-2.8-r0 do_package_qa: QA Issue:
nativesdk-bash-completion rdepends on bash, but it isn't a build
dependency? [build-deps]

with bitbake 5463c16e3619d324aed137f47f93f0997a227d29
# $RDEPENDS_ca-certificates [4 operations]
#   _remove[pn-iputils] conf/distro/include/some-distro.inc:52
#     " ${PN}-ping6"
#   append oe-core/meta/recipes-support/ca-certificates/
ca-certificates_20180409.bb:85
#     "openssl"
#   _remove
meta-webos/recipes-support/ca-certificates/ca-certificates_%.bbappend:24
#     "openssl"
#   rename from RDEPENDS_${PN} data.py:117 [expandKeys]
#     " openssl"
# pre-expansion value:
#   " openssl"
RDEPENDS_ca-certificates=" "

and do_package_qa sees the openssl in rdepends:

NOTE: recipe ca-certificates-20180409-r0: task do_package_qa: Started
WARNING: ca-certificates-20180409-r0 do_package_qa: build-dep:
ca-certificates-dev
WARNING: ca-certificates-20180409-r0 do_package_qa: build-dep: openssl
WARNING: ca-certificates-20180409-r0 do_package_qa: rdep_data for rdepend:
openssl are {}
WARNING: ca-certificates-20180409-r0 do_package_qa: trying
/jenkins/mjansa/build-signage-master/BUILD/pkgdata/m16pp/runtime-rprovides/openssl/
ERROR: ca-certificates-20180409-r0 do_package_qa: QA Issue: ca-certificates
rdepends on openssl, but it isn't a build dependency? [build-deps]
WARNING: ca-certificates-20180409-r0 do_package_qa: build-dep:
ca-certificates
ERROR: ca-certificates-20180409-r0 do_package_qa: QA run found fatal
errors. Please consider fixing them.
ERROR: ca-certificates-20180409-r0 do_package_qa: Function failed:
do_package_qa
ERROR: Logfile of failure stored in:
/jenkins/mjansa/build-signage-master/BUILD/work/all-signage-linux/ca-certificates/20180409-r0/temp/log.do_package_qa.46878
NOTE: recipe ca-certificates-20180409-r0: task do_package_qa: Failed

with bitbake a68de8ace62eaba23856bfb301efbbe1824322aa
# $RDEPENDS_ca-certificates [4 operations]
#   _remove[pn-iputils] conf/distro/include/some-distro.inc:52
#     " ${PN}-ping6"
#   append oe-core/meta/recipes-support/ca-certificates/
ca-certificates_20180409.bb:85
#     "openssl"
#   _remove
meta-webos/recipes-support/ca-certificates/ca-certificates_%.bbappend:24
#     "openssl"
#   rename from RDEPENDS_${PN} data.py:116 [expandKeys]
#     " "
# pre-expansion value:
#   " "
RDEPENDS_ca-certificates=" "

git log --oneline a68de8a..5463c16e | tee
5463c16e test/data: Add new tests for task checksum changing/not changing
796a20d2 data/siggen: Extract task hash generation code into a function
57d2ee17 bitbake: data: Ensure task checksums account for remove data
136100dc data/data_smart: Allow getVarFlag to return the variable parser
object
a039052f data_smart: Fix expand_cache and _remove operator interaction
issues
6d19eb32 data_smart: Micro optimise _remove handling

With the micro optimise commit 6d19eb32 it already has openssl in "rename
from RDEPENDS_${PN}":
# $RDEPENDS_ca-certificates [4 operations]
#   _remove[pn-iputils] conf/distro/include/some-disto.inc:52
#     " ${PN}-ping6"
#   append oe-core/meta/recipes-support/ca-certificates/
ca-certificates_20180409.bb:85
#     "openssl"
#   _remove
meta-webos/recipes-support/ca-certificates/ca-certificates_%.bbappend:24
#     "openssl"
#   rename from RDEPENDS_${PN} data.py:116 [expandKeys]
#     " openssl"
# pre-expansion value:
#   " "
RDEPENDS_ca-certificates=" "

The bbappend doesn't do anything special, just this:
# ca-certificates is allarch, so it either has to RDEPENDS on
${MLPREFIX}openssl or
# in this case it's easier to just remove the dependency and pull it from
somewhere
# else
RDEPENDS_${PN}_remove = "openssl"

and similar _remove bbappend for bash-completion:
VIRTUAL-RUNTIME_bash ?= "bash"
RDEPENDS_${PN}_append_class-target = " ${VIRTUAL-RUNTIME_bash}"
RDEPENDS_${PN}_remove = "bash"

The reason for bash-completion is probably the same as in ca-certificates,
but it's surprising that I have maybe a 100 of bbappends like above with
bash, but only these 2 are causing the QA error.

Will dig a bit more where exactly it was introduced and will try to
reproduce with just oe-core.

On Thu, Oct 18, 2018 at 11:23 AM Richard Purdie <
richard.purdie at linuxfoundation.org> wrote:

> This brings _remove handling into line with _append/_prepend with regard
> to the parsing flag to getVarFlag.
>
> This is an internal flag and the only times this is used is through getVar
> during renameVar operations and when processing ?= operations to see if
> a variable is set. In either case we don't need to process remove
> operations.
>
> Therefore take the minor speedup and skip processing for parsing=True.
>
> [YOCTO #10945]
>
> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> ---
>  lib/bb/data_smart.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> index 0a8488ca1b..7b2c0a8943 100644
> --- a/lib/bb/data_smart.py
> +++ b/lib/bb/data_smart.py
> @@ -805,7 +805,7 @@ class DataSmart(MutableMapping):
>                  cachename = var + "[" + flag + "]"
>              value = self.expand(value, cachename)
>
> -        if value and flag == "_content" and local_var is not None and
> "_remove" in local_var:
> +        if value and flag == "_content" and local_var is not None and
> "_remove" in local_var and not parsing:
>              removes = []
>              self.need_overrides()
>              for (r, o) in local_var["_remove"]:
> --
> 2.17.1
>
> --
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20181019/9ca4818d/attachment-0002.html>