[OE-core] [oe-commits] Robert Yang : A script to clean obsolete sstate cache files

Tue Mar 27 13:28:09 UTC 2012

Hi Martin,

Thanks for reporting this, and I'm very sorry for the inconvenience,
please see my comment inline ...

On 03/27/2012 07:33 PM, Martin Jansa wrote:
> On Tue, Mar 27, 2012 at 01:06:33PM +0200, Martin Jansa wrote:
>> On Fri, Feb 24, 2012 at 12:01:37AM +0000, git at git.openembedded.org wrote:
>>> Module: openembedded-core.git
>>> Branch: master
>>> Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
>>> URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
>>>
>>> Author: Robert Yang<liezhi.yang at windriver.com>
>>> Date:   Wed Feb 22 20:38:30 2012 +0800
>>>
>>> A script to clean obsolete sstate cache files
>>>
>>
>>> +# Use the "_" to substitute "-", e.g., x86-64 to x86_64
>>> +all_archs=`grep -r DEFAULTTUNE $tunedir | \
>>> +    sed -e 's/.*\"\(.*\)\"/\1/' -e 's/-/_/g' | sort -u`
>>> +# Add the qemu archs
>>> +all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
>>
>> This doesn't list any thumb variants, current list is:
>>
>> arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a
>> core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586
>> iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc
>> ppce500v2 ppce5500 strongarm x86 x86_64 xscale
>>
>> so thumb versions (e.g. armv4t) are not found in
>>
>>> +for arch in $all_archs; do
>>> +    ls | grep -q -w $arch
>>> +    [ $? -eq 0 ]&&  ava_archs="$ava_archs $arch"
>>> +done
>>
>> Cheers,
>
> And after extending list of machines in all_archs with armv4t and my
> machines I got only one file in sstate-cache left :/
>
> First run without armv4t found:
> OE crespo at shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
> 14G     sstate-cache/
> Figuring out the archs in the sstate cache dir ...
> The following archs have been found in the sstate cache dir:
> armv7a x86_64
> Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
> Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
> Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
> Removing the sstate-xxx_deploy.tgz ... (0 files)
> Removing the sstate-xxx_package.tgz ... (62 files)
> Removing the sstate-xxx_populate-lic.tgz ... (46 files)
> Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
> 268 files have been removed
> 13G     sstate-cache/
>
> Second run with armv4t manually added
> -all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
> +all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
> OE crespo at shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
> 13G     ../shr-core/sstate-cache/
> /OE/openembedded-core/meta/conf/machine/include
> Figuring out the archs in the sstate cache dir ...
> The following archs have been found in the sstate cache dir:
> armv4t armv7a x86_64
> Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
> Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
> Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
> Removing the sstate-xxx_deploy.tgz ... (0 files)
> Removing the sstate-xxx_package.tgz ... (454 files)
> Removing the sstate-xxx_populate-lic.tgz ... (456 files)
> Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
> 1838 files have been removed
> 6.2G    ../shr-core/sstate-cache/
>
> Third run with machines I'm building manually added
> -all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
> +all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"

The script figures the archs according to the 'meta/conf/machine/include',
this is incorrect since it only searches the oe-core layer, I will fix it.

And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which
layer are they in? So that I can test them.

> OE crespo at shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
> 6.2G    ../shr-core/sstate-cache/
> /OE/openembedded-core/meta/conf/machine/include
> Figuring out the archs in the sstate cache dir ...
> The following archs have been found in the sstate cache dir:
> armv4t armv7a x86_64 nokia900 palmpre palmpre2 crespo
> Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
> Removing the sstate-xxx_deploy-ipk.tgz ... (7024 files)
> Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
> Removing the sstate-xxx_deploy.tgz ... (0 files)
> Removing the sstate-xxx_package.tgz ... (0 files)
> Removing the sstate-xxx_populate-lic.tgz ... (0 files)
> Removing the sstate-xxx_populate-sysroot.tgz ... (0 files)
> 7024 files have been removed
> 3.1M    ../shr-core/sstate-cache/

This seems terrible, I don't know what's wrong currently, I will try to fix it 
tomorrow.

// Robert

>
> OE crespo at shr ~/openembedded-core $ ls ../shr-core/sstate-cache/
> sstate-shr-version-crespo-oe-linux-gnueabi-1.020120327-r0-crespo-2-fcb5bb2c0ac03d75285d77e7c30af869_populate-sysroot.tgz
>
> Original count of sstate packages was something like:
> Sum                     3043
> all-oe-linux            661
>      gnueabi              94
> nativesdk               953 (i686 + x86_64)
>    x86_64-nativesdk      480
>      gnueabi              37
>    i686-nativesdk        473
>      gnueabi              36
> x86_64-linux            309
> nokia900                229
>      gnueabi             222
> crespo                  215
>      gnueabi             211
> palmpre                 475 (palmpre + palmpre2)
>    palmpre               244
>      gnueabi             236
>    palmpre2              231
>      gnueabi             223
> om_gta04                201
>      gnueabi             193
>
> Cheers,
>