[OE-core] [oe-commits] Robert Yang : A script to clean obsolete sstate cache files

Tue Mar 27 15:19:33 UTC 2012

On Tue, Mar 27, 2012 at 09:49:36PM +0800, Robert Yang wrote:
> 
> 
> On 03/27/2012 09:35 PM, Martin Jansa wrote:
> > On Tue, Mar 27, 2012 at 09:28:09PM +0800, Robert Yang wrote:
> >>
> >> Hi Martin,
> >>
> >> Thanks for reporting this, and I'm very sorry for the inconvenience,
> >> please see my comment inline ...
> >>
> >> On 03/27/2012 07:33 PM, Martin Jansa wrote:
> >>> On Tue, Mar 27, 2012 at 01:06:33PM +0200, Martin Jansa wrote:
> >>>> On Fri, Feb 24, 2012 at 12:01:37AM +0000, git at git.openembedded.org wrote:
> >>>>> Module: openembedded-core.git
> >>>>> Branch: master
> >>>>> Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
> >>>>> URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
> >>>>>
> >>>>> Author: Robert Yang<liezhi.yang at windriver.com>
> >>>>> Date:   Wed Feb 22 20:38:30 2012 +0800
> >>>>>
> >>>>> A script to clean obsolete sstate cache files
> >>>>>
> >>>>
> >>>>> +# Use the "_" to substitute "-", e.g., x86-64 to x86_64
> >>>>> +all_archs=`grep -r DEFAULTTUNE $tunedir | \
> >>>>> +    sed -e 's/.*\"\(.*\)\"/\1/' -e 's/-/_/g' | sort -u`
> >>>>> +# Add the qemu archs
> >>>>> +all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
> >>>>
> >>>> This doesn't list any thumb variants, current list is:
> >>>>
> >>>> arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a
> >>>> core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586
> >>>> iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc
> >>>> ppce500v2 ppce5500 strongarm x86 x86_64 xscale
> >>>>
> >>>> so thumb versions (e.g. armv4t) are not found in
> >>>>
> >>>>> +for arch in $all_archs; do
> >>>>> +    ls | grep -q -w $arch
> >>>>> +    [ $? -eq 0 ]&&   ava_archs="$ava_archs $arch"
> >>>>> +done
> >>>>
> >>>> Cheers,
> >>>
> >>> And after extending list of machines in all_archs with armv4t and my
> >>> machines I got only one file in sstate-cache left :/
> >>>
> >>> First run without armv4t found:
> >>> OE crespo at shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
> >>> 14G     sstate-cache/
> >>> Figuring out the archs in the sstate cache dir ...
> >>> The following archs have been found in the sstate cache dir:
> >>> armv7a x86_64
> >>> Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
> >>> Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
> >>> Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
> >>> Removing the sstate-xxx_deploy.tgz ... (0 files)
> >>> Removing the sstate-xxx_package.tgz ... (62 files)
> >>> Removing the sstate-xxx_populate-lic.tgz ... (46 files)
> >>> Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
> >>> 268 files have been removed
> >>> 13G     sstate-cache/
> >>>
> >>> Second run with armv4t manually added
> >>> -all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
> >>> +all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
> >>> OE crespo at shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
> >>> 13G     ../shr-core/sstate-cache/
> >>> /OE/openembedded-core/meta/conf/machine/include
> >>> Figuring out the archs in the sstate cache dir ...
> >>> The following archs have been found in the sstate cache dir:
> >>> armv4t armv7a x86_64
> >>> Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
> >>> Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
> >>> Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
> >>> Removing the sstate-xxx_deploy.tgz ... (0 files)
> >>> Removing the sstate-xxx_package.tgz ... (454 files)
> >>> Removing the sstate-xxx_populate-lic.tgz ... (456 files)
> >>> Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
> >>> 1838 files have been removed
> >>> 6.2G    ../shr-core/sstate-cache/
> >>>
> >>> Third run with machines I'm building manually added
> >>> -all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
> >>> +all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
> >>
> >> The script figures the archs according to the 'meta/conf/machine/include',
> >> this is incorrect since it only searches the oe-core layer, I will fix it.
> >
> > The first issue is that armv4t and other thumb variants are also in
> > oe-core layer.
> >
> >> And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which
> >> layer are they in? So that I can test them.
> >
> > Those are machines from meta-smartphone sublayers
> > http://git.shr-project.org/git/?p=meta-smartphone.git;a=summary
> >
> > Maybe add some param to add extra archs/machines or just list all used
> > xxx variants from 'sstate-xxx_'.
> >
> Thanks, add some param is good choice. Figure out the used variants from
> sstate-xxx is the best way, but it seems not easy since it uses the "-" as
> the separator, and I had counted the number of field, it is not always the
> same.

I see, maybe add also param to specify stamps dir, which can be used to
find all arch ever built if it's the same host which populated
sstate-cache dir.

That would include also all sdk and native variants etc.

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20120327/63e75f04/attachment-0002.sig>