[OE-core] Fetch time optimization (svn : gcc/eglibc - git : linux-yocto)

[Martin Jansa]

On Thu, Mar 29, 2012 at 09:03:15PM -0400, Bruce Ashfield wrote:
> On Thu, Mar 29, 2012 at 6:03 PM, Richard Purdie
> <richard.purdie at linuxfoundation.org> wrote:
> > On Thu, 2012-03-29 at 22:53 +0200, Eric Bénard wrote:
> >> I noticed in from scratch builds for qemuarm that the longest time is
> >> taken in fetching sources, especially those fetched using git
> >> (linux-yocto for example) & svn (gcc, eglibc & co).
> >
> > Are you timing these as fetches from the source control systems or from
> > the mirror tarballs of the repositories. The tarballs should be
> > faster...
> >
> >> To reduce the fetch time would that make sense to
> >> - fetch gcc/glibc & co from the archive of a stable version and then
> >>   apply patches on top of it (maybe patches stored in an archive
> >>   fetched from oe's website and applied in bulk or patches stored in OE)
> >
> > Unfortunately the patches tend to get unwieldy. The tarballs of the svn
> > repos on the mirror should be about equal in size to the upstream
> > archive in this case.
> >
> >> - do the same thing for the linux-yocto kernel or add a --reference
> >>   option to the git fetcher so that we can provide a local tree as a
> >>   reference ?
> >
> > This is effectively how the repositories in DL_DIR are used. If you
> > place a tree in the right place there, it should reuse references...
> 
> Agreed .. they definitely do here.

What's right place?

I guess the idea was to use --reference for e.g. some other kernel recipe
sources checkout.

And I guess that building linux-foo won't notice that there is e.g.
/OE/downloads/git2/gitorious.org.shr.linux.git:
from which it can share a lot of objects using --reference

Bob Ham (rah on #oe) said that he is working on some sort of support
for --reference with bitbake after I've refused to add just another
linux-bar recipe to meta-smartphone, but not sure how he plans to
implement it to be usefull and working oob in different env with
different sources available in downloads dir.

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20120330/a0b2a189/attachment-0002.sig>