[OE-core] The design document for ccache-native

[Mike Westerhof]

On 6/10/2011 8:09 AM, Mike Westerhof wrote:
> On 6/10/2011 12:48 AM, Robert Yang wrote:
>>
>> On 06/10/2011 11:26 AM, wenzong fan wrote:
>>>
>>>
>>> -------- Original Message --------
>>>
>>> On Thu, 2011-06-09 at 15:40 -0700, Saul Wold wrote:
>>>> On 06/02/2011 08:11 PM, wenzong fan wrote:
>>>>> Hi Folks,
>>>>>
>>>>> Please help me to review the design document for ccache-native, and
>>>>> I also have two questions about it, any answers or suggestions are
>>>>> appreciated.
>>>>>
>>>>> * Feature name: ccache-native
>>>>> Priority: P3; M2
>>>>> Owner: Wenzong Fan
>>>>> Summary: Integrate ccache-native to yocto
>>>>>
>>>>> * Description:
>>>>> Bitbake supports the 'CCACHE Mechanism', but 'ccache' hasn't been
>>>>> included by poky/yocto, just add it as a native tool.
>>>>>
>>>>> * Usage:
>>>>> Build ccache as a native tool by default and enable it for speeding
>>>>> target packages build.
>>>>>
>>>>> * Implementation:
>>>>> 1) Copy bb file from OE upstream to:
>>>>> meta/recipes-devtools/ccache/
>>>>>
>>>>> 2) Update bb file to get the latest ccache_3.1.5 and split the single
>>>>> bb file to:
>>>>> 'ccache_3.1.5.bb', 'ccache.inc'
>>>>>
>>>>> 3) Enable ccache in the native tools building.
>>>>>
>>>> You will need to have it be a dependency pretty early on in the build.
>>>> Additionally, this is a bit a new part to this task, we want to have the
>>>> default CCACHE_DIR for the build default to a directory in TMPDIR
>>>> instead of the user's home directory. This will mean setting an
>>>> environment variable somewhere early also.
>>>
>>> There is a little more detail on:
>>>
>>> https://wiki.yoctoproject.org/wiki/Yocto_1.1_Schedule
>>>
>>> Specifically, "c) Set CCACHE on a per recipe basis. need to figure out
>>> whether ccache data can be shared and under what circumstances."
>>>
>>> so something like adding:
>>>
>>> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/${PN}"
>>>
>>
>> I think that set CCACHE_DIR on per recipe basis would degrade hit
>> efficiency,
>> the following ccache data can be shared if they they use the same
>> CCACHE_DIR,
>> but if we set CACHE_DIR on a per recipe basis, then they can't be shared:
>>
>> 1) Most pkg's configure will run "gcc foo.c" for checking the C  compiler,
>>    these compiling are the same between different pkgs at most time.
>>
>> 2) Some recipes' compiling are similar, for example: gcc-cross,
>>    gcc-corss-initial and gcc-cross-intermediate.
>>
>> I think that:
>>
>> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/"
>>
>> would be better.
> 
> I fear that the use of ccache is inherently risky with OE.  Given the
> (relatively common) case where the user blows away their TMPDIR in order

(Replying to my own message, because I didn't have enough coffee to be
clear in the original!)  I'm agreeing with the suggestion that it be in
TMPDIR, and offering the suggestion that if we don't put it there, we
need to qualify it with the toolchain version.  (Now I'm going to get
another cup of coffee -- sorry for any confusion!)
-Mike (mwester)

> to get a full, clean rebuild after an update to the toolchain is make,
> it is possible that ccache will erroneously re-use an object created by
> the earlier version of the toolchain.
> 
> While imperfect, I would suggest that we would do better if we would
> somehow embed the PV (or something like that) for the toolchain into the
> CCACHE_DIR, thus ensuring that we don't risk the re-use of old objects
> when the toolchain is updated.
> 
> (Frankly, given my experiences with it, I'd prefer we just disable
> ccache entirely with OE.)
> 
> -Mike (mwester)
> 
>> // Robert
>>
>>> to bitbake.conf with a bit more thought into working out the right
>>> components to add to the variable.
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>>
>>>
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>>
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> 
>