[bitbake-devel] Issue with "_remove" override and variables

[Ricardo Silva]

Hi All,

In order to reduce the final image size, I want to remove some unneeded runtime
dependencies from some python3 packages, since I know that I won't be needing
them and they have a big impact on the image size.

My idea was to add a bbappend file to my layer in order to remove the
dependencies I want from the base recipe.

The base recipe has runtime dependencies like:

    RDEPENDS_${PN} += " \
        ${PYTHON_PN}-email \
        ${PYTHON_PN}-json \
        ${PYTHON_PN}-ndg-httpsclient \
        ${PYTHON_PN}-netserver \
        ${PYTHON_PN}-pyasn1 \
        ${PYTHON_PN}-pyopenssl \
        ${PYTHON_PN}-pysocks \
        ${PYTHON_PN}-urllib3 \
    "

Imagine I would like to remove the dependency from pyopenssl. I would add a
bbappend file in my layer for the base recipe and use, e.g.:

    RDEPENDS_${PN}_remove = "${PYTHON_PN}-pyopenssl"

I observe that this is not working as I was expecting, as the pyopenssl package
still gets installed, and there's no other package that depends on it. In fact,
if I just fully override RDEPENDS in the bbappend file to:

    RDEPENDS_${PN} = " \
        ${PYTHON_PN}-email \
        ${PYTHON_PN}-json \
        ${PYTHON_PN}-ndg-httpsclient \
        ${PYTHON_PN}-netserver \
        ${PYTHON_PN}-pyasn1 \
        ${PYTHON_PN}-pysocks \
        ${PYTHON_PN}-urllib3 \
    "

i.e., to have all dependencies of the base recipe except the one I want to
remove, it does work.

I think this might be related to variable expansion, because if I change the
base recipe's RDEPENDS to the following, the RDEPENDS_${PN}_remove does work:

    RDEPENDS_${PN} += " \
        ${PYTHON_PN}-email \
        ${PYTHON_PN}-json \
        ${PYTHON_PN}-ndg-httpsclient \
        ${PYTHON_PN}-netserver \
        ${PYTHON_PN}-pyasn1 \
        python3-pyopenssl \
        ${PYTHON_PN}-pysocks \
        ${PYTHON_PN}-urllib3 \
    "

I've tried checking the parsing of RDEPENDS by using `bitbake -e` on the
relevant recipe and I do notice some differences between the 3 cases:

(1) Base recipe only, no bbappend file for removing pyopenssl:

    # $RDEPENDS_python3-requests
    #   rename from RDEPENDS_${PN} data.py:116 [expandKeys]
    #     " ${@['', '${PYTHON_PN}-core']['${CLASSOVERRIDE}' == 'class-target']}      ${PYTHON_PN}-email     ${PYTHON_PN}-json     ${PYTHON_PN}-ndg-httpsclient     ${PYTHON_PN}-netserver     ${PYTHON_PN}-pyasn1     ${PYTHON_PN}-pyopenssl     ${PYTHON_PN}-pysocks     ${PYTHON_PN}-urllib3  ${PYTHON_PN}-misc"
    RDEPENDS_python3-requests=" python3-core      python3-email     python3-json     python3-ndg-httpsclient     python3-netserver     python3-pyasn1     python3-pyopenssl     python3-pysocks     python3-urllib3  python3-misc"

    Note that pyopenssl appears on last 2 lines.

(2) With RDEPENDS_${PN}_remove failing to remove pyopenssl:

    # $RDEPENDS_python3-requests
    #   rename from RDEPENDS_${PN} data.py:116 [expandKeys]
    #     "${@['', '${PYTHON_PN}-core']['${CLASSOVERRIDE}' == 'class-target']} ${PYTHON_PN}-email ${PYTHON_PN}-json ${PYTHON_PN}-ndg-httpsclient ${PYTHON_PN}-netserver ${PYTHON_PN}-pyasn1 ${PYTHON_PN}-pyopenssl ${PYTHON_PN}-pysocks ${PYTHON_PN}-urllib3 ${PYTHON_PN}-misc"
    RDEPENDS_python3-requests="python3-core python3-email python3-json python3-ndg-httpsclient python3-netserver python3-pyasn1 python3-pysocks python3-urllib3 python3-misc"

    Note that pyopenssl now appears only on the line before last.

(3) With RDEPENDS_${PN}_remove succeeding to remove pyopenssl (because base
    recipe was changed to have "python3-pyopenssl" instead of
    "${PYTHON_PN}-pyopenssl"):

    # $RDEPENDS_python3-requests
    #   rename from RDEPENDS_${PN} data.py:116 [expandKeys]
    #     "${@['', '${PYTHON_PN}-core']['${CLASSOVERRIDE}' == 'class-target']} ${PYTHON_PN}-email ${PYTHON_PN}-json ${PYTHON_PN}-ndg-httpsclient ${PYTHON_PN}-netserver ${PYTHON_PN}-pyasn1 ${PYTHON_PN}-pysocks ${PYTHON_PN}-urllib3 ${PYTHON_PN}-misc"
    RDEPENDS_python3-requests="python3-core python3-email python3-json python3-ndg-httpsclient python3-netserver python3-pyasn1 python3-pysocks python3-urllib3 python3-misc"

    Note now that there's no reference at all to pyopenssl.

I guess this is most likely some misuse on my side of variables attribution
(immediate vs deferred, etc), but from the documentation and other similar
examples I've searched out there, I could not yet find what's wrong, so any help
would be highly appreciated.

I'm running bitbake "BitBake Build Tool Core version 1.34.0" in commit
ca9b9ffc250e ("toaster: allow dots in user path names").

I've found a similar issue being described in the bitbake-devel list archives
and stackoverflow [1], but I don't think it's the same issue, because I have
already the fix for the described problem and in my case the problem is when a
variable is used in the text to be replaced, not in the text to replace.

[1] https://stackoverflow.com/questions/35336111/bitbake-override-style-removal-with-variable-expansion

Thanks in advance,

Ricardo Silva