[OE-core] The design document for ccache-native

[Mike Westerhof]

On 6/10/2011 12:48 AM, Robert Yang wrote:
> 
> On 06/10/2011 11:26 AM, wenzong fan wrote:
>>
>>
>> -------- Original Message --------
>>
>> On Thu, 2011-06-09 at 15:40 -0700, Saul Wold wrote:
>>> On 06/02/2011 08:11 PM, wenzong fan wrote:
>>> > Hi Folks,
>>> >
>>> > Please help me to review the design document for ccache-native, and
>>> > I also have two questions about it, any answers or suggestions are
>>> > appreciated.
>>> >
>>> > * Feature name: ccache-native
>>> > Priority: P3; M2
>>> > Owner: Wenzong Fan
>>> > Summary: Integrate ccache-native to yocto
>>> >
>>> > * Description:
>>> > Bitbake supports the 'CCACHE Mechanism', but 'ccache' hasn't been
>>> > included by poky/yocto, just add it as a native tool.
>>> >
>>> > * Usage:
>>> > Build ccache as a native tool by default and enable it for speeding
>>> > target packages build.
>>> >
>>> > * Implementation:
>>> > 1) Copy bb file from OE upstream to:
>>> > meta/recipes-devtools/ccache/
>>> >
>>> > 2) Update bb file to get the latest ccache_3.1.5 and split the single
>>> > bb file to:
>>> > 'ccache_3.1.5.bb', 'ccache.inc'
>>> >
>>> > 3) Enable ccache in the native tools building.
>>> >
>>> You will need to have it be a dependency pretty early on in the build.
>>> Additionally, this is a bit a new part to this task, we want to have the
>>> default CCACHE_DIR for the build default to a directory in TMPDIR
>>> instead of the user's home directory. This will mean setting an
>>> environment variable somewhere early also.
>>
>> There is a little more detail on:
>>
>> https://wiki.yoctoproject.org/wiki/Yocto_1.1_Schedule
>>
>> Specifically, "c) Set CCACHE on a per recipe basis. need to figure out
>> whether ccache data can be shared and under what circumstances."
>>
>> so something like adding:
>>
>> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/${PN}"
>>
> 
> I think that set CCACHE_DIR on per recipe basis would degrade hit
> efficiency,
> the following ccache data can be shared if they they use the same
> CCACHE_DIR,
> but if we set CACHE_DIR on a per recipe basis, then they can't be shared:
> 
> 1) Most pkg's configure will run "gcc foo.c" for checking the C  compiler,
>    these compiling are the same between different pkgs at most time.
> 
> 2) Some recipes' compiling are similar, for example: gcc-cross,
>    gcc-corss-initial and gcc-cross-intermediate.
> 
> I think that:
> 
> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/"
> 
> would be better.

I fear that the use of ccache is inherently risky with OE.  Given the
(relatively common) case where the user blows away their TMPDIR in order
to get a full, clean rebuild after an update to the toolchain is make,
it is possible that ccache will erroneously re-use an object created by
the earlier version of the toolchain.

While imperfect, I would suggest that we would do better if we would
somehow embed the PV (or something like that) for the toolchain into the
CCACHE_DIR, thus ensuring that we don't risk the re-use of old objects
when the toolchain is updated.

(Frankly, given my experiences with it, I'd prefer we just disable
ccache entirely with OE.)

-Mike (mwester)

> // Robert
> 
>> to bitbake.conf with a bit more thought into working out the right
>> components to add to the variable.
>>
>> Cheers,
>>
>> Richard
>>
>>
>>
>>
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> 
>