[OE-core] [PATCH V3 03/11] ccache.bbclass: Refactor it to make it more reliable
Robert Yang
liezhi.yang at windriver.com
Tue Jan 15 02:11:05 UTC 2019
Hi Randy,
I'm sorry for the types, and thanks for the correction. I found that it has
been merged, I will be more careful.
// Robert
On 1/12/19 4:49 AM, Randy MacLeod wrote:
> A couple of typos and grammar improvements, otherwise a good
> improvement for iterative development of a package/image where
> sstate isn't useful.
>
> On 12/18/18 2:18 AM, Robert Yang wrote:
>> The previous ccache.bbclass has the following problems:
>> - It uses host's ccache for native recipes, but this may not work on some
>> hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
>> build failures (m4-native failed at do_configure, and others will also be
>> failed if I disable CCACHE for m4-native)
>>
>> - native/nativesdk/cross/crosssdk recipes use host's ccahe, but target uses
> * host's ccache
>> ccache-native, this may confuse user.
>>
>> - The target recipes may use both host's ccache and ccache-native, this may
>> cause unexpected problems and hard to debug. This is because ccache-native is
> * and be hard to debug
>> in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
>> rebuild target recipes, and then it would use hosttools/ccache, but the
>> previous ccache files were generated by ccache-native.
>>
>> - Target recipes can't use ccache when no ccahe is installed on the host:
> * no ccache
>
>> CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
>>
>> After refactored:
>> All types recipes (native, target and others) will use ccache-native except
>> ccache-native itself, host's cache won't be used any more. It is more
>> reliable now, which will work everywhere when ccache-native can be built.
>>
>> And now we need use "CCACHE_DISABLE = '1'" to disable ccahe for the recipe
> * disable ccache
>
> ../Randy
>
>
>> rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
>> d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
>> can't check whether CCACHE is set or not in anonymous function since it is
>> always set. Use CCACHE_DISABLE to disable it would be more clear.
>>
>> Signed-off-by: Robert Yang <liezhi.yang at windriver.com>
>> ---
>> meta/classes/ccache.bbclass | 25 ++++++++++++++++++++++---
>> meta/conf/bitbake.conf | 6 +-----
>> meta/conf/layer.conf | 1 -
>> meta/lib/oe/utils.py | 3 +++
>> 4 files changed, 26 insertions(+), 9 deletions(-)
>>
>> diff --git a/meta/classes/ccache.bbclass b/meta/classes/ccache.bbclass
>> index 9609020..59e1022 100644
>> --- a/meta/classes/ccache.bbclass
>> +++ b/meta/classes/ccache.bbclass
>> @@ -1,4 +1,14 @@
>> -CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"
>> +#
>> +# Usage:
>> +# - Enable ccache
>> +# Add the following line to a conffile such as conf/local.conf:
>> +# INHERIT += "ccache"
>> +#
>> +# - Disable ccache for a recipe
>> +# Add the following line to the recipe if it can't be built with ccache:
>> +# CCACHE_DISABLE = '1'
>> +#
>> +
>> export CCACHE_DIR ?= "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>> # We need to stop ccache considering the current directory or the
>> @@ -7,5 +17,14 @@ export CCACHE_DIR ?=
>> "${TMPDIR}/ccache/${MULTIMACH_TARGET_SYS}/${PN}"
>> # ${PV} or ${PR} change.
>> export CCACHE_NOHASHDIR ?= "1"
>> -DEPENDS_append_class-target = " ccache-native"
>> -DEPENDS[vardepvalueexclude] = " ccache-native"
>> +python() {
>> + """
>> + Enable ccache for the recipe
>> + """
>> + pn = d.getVar('PN')
>> + # quilt-native doesn't need ccache since no c files
>> + if not (pn in ('ccache-native', 'quilt-native') or
>> + bb.utils.to_boolean(d.getVar('CCACHE_DISABLE'))):
>> + d.appendVar('DEPENDS', ' ccache-native')
>> + d.setVar('CCACHE', 'ccache ')
>> +}
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index 6480062..c88d66c 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -492,7 +492,7 @@ HOSTTOOLS += " \
>> HOSTTOOLS += "${@'ip ping ps scp ssh stty' if
>> (bb.utils.contains_any('IMAGE_CLASSES', 'testimage testsdk', True, False, d)
>> or any(x in (d.getVar("BBINCLUDED") or "") for x in ["testimage.bbclass",
>> "testsdk.bbclass"])) else ''}"
>> # Link to these if present
>> -HOSTTOOLS_NONFATAL += "aws ccache gcc-ar gpg ld.bfd ld.gold nc pigz sftp
>> socat ssh sudo"
>> +HOSTTOOLS_NONFATAL += "aws gcc-ar gpg ld.bfd ld.gold nc pigz sftp socat ssh
>> sudo"
>> # Temporary add few more detected in bitbake world
>> HOSTTOOLS_NONFATAL += "join nl size yes zcat"
>> @@ -504,10 +504,6 @@ HOSTTOOLS_NONFATAL += "bzr"
>> HOSTTOOLS_NONFATAL += "scp"
>> CCACHE ??= ""
>> -# ccache < 3.1.10 will create CCACHE_DIR on startup even if disabled, and
>> -# autogen sets HOME=/dev/null so in certain situations builds can fail.
>> -# Explicitly export CCACHE_DIR until we can assume ccache >3.1.10 on the host.
>> -export CCACHE_DIR ??= "${@os.getenv('HOME')}/.ccache"
>> TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
>> diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf
>> index c67dec4..4141d6c 100644
>> --- a/meta/conf/layer.conf
>> +++ b/meta/conf/layer.conf
>> @@ -49,7 +49,6 @@ SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS += " \
>> *->quilt-native \
>> *->subversion-native \
>> *->git-native \
>> - *->ccache-native \
>> *->icecc-create-env-native \
>> gcc-cross-${TARGET_ARCH}->virtual/${TARGET_PREFIX}libc-for-gcc \
>> gcc-cross-${TARGET_ARCH}->linux-libc-headers \
>> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
>> index 29b4115..ee6f0e6 100644
>> --- a/meta/lib/oe/utils.py
>> +++ b/meta/lib/oe/utils.py
>> @@ -363,6 +363,9 @@ def host_gcc_version(d, taskcontextonly=False):
>> return
>> compiler = d.getVar("BUILD_CC")
>> + # Get rid of ccache since it is not present when parsing.
>> + if compiler.startswith('ccache '):
>> + compiler = compiler[7:]
>> try:
>> env = os.environ.copy()
>> env["PATH"] = d.getVar("PATH")
>>
>
>
More information about the Openembedded-core
mailing list