<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jun 2, 2016 at 12:24 PM, Paul Eggleton <span dir="ltr"><<a href="mailto:paul.eggleton@linux.intel.com" target="_blank">paul.eggleton@linux.intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div><div>On Thu, 02 Jun 2016 15:19:37 Bruce Ashfield wrote:<br>
> On 2016-06-02 3:15 PM, Paul Eggleton wrote:<br>
> > On Thu, 02 Jun 2016 18:05:35 Martin Jansa wrote:<br>
> >> On Wed, Jun 01, 2016 at 10:23:35AM +1200, Paul Eggleton wrote:<br>
> >>> On Wed, 01 Jun 2016 10:20:23 Paul Eggleton wrote:<br>
> >>>> On Tue, 31 May 2016 11:12:21 Jussi Kukkonen wrote:<br>
> >>>>> On 11 May 2016 at 20:35, Khem Raj <<a href="mailto:raj.khem@gmail.com" target="_blank">raj.khem@gmail.com</a>> wrote:<br>
> >>>>>> +#BASEURI ?= "${GNU_MIRROR}/gcc/gcc-${PV}/gcc-${PV}.tar.bz2"<br>
> >>>>>> +BASEURI ?= "git://<br>
> >>>>>> <a href="http://github.com/gcc-mirror/gcc;branch=gcc-6-branch;protocol=git" rel="noreferrer" target="_blank">github.com/gcc-mirror/gcc;branch=gcc-6-branch;protocol=git</a>"<br>
> >>>>><br>
> >>>>> I guess this is where git2_github.com.gcc-mirror.gcc.tar.gz download<br>
> >>>>> comes from? It increased the size of my downloads-directory by >30% --<br>
> >>>>> and I must have quite a bit of old junk in that directory already.<br>
> >>>>><br>
> >>>>> Is there no other solution to this than a 2.5G git copy (honest<br>
> >>>>> question, I don't know gcc)?<br>
> >>>><br>
> >>>> We went down this road before and it wasn't great for users. There is<br>
> >>>> at<br>
> >>>> least a tarball for 6.1, we'd presumably need some patches on top of<br>
> >>>> that (~43 if we were to simply take what's in the gcc-6-branch between<br>
> >>>> 6.1 and bd9a826, minus the "daily bumps").<br>
> >>><br>
> >>> Perhaps that was a little unclear - when I say "we went down this road<br>
> >>> before" I'm agreeing with Jussi - we pointed to a git branch in a<br>
> >>> previous release with the same resulting huge download for everyone,<br>
> >>> something I think we should avoid if at all possible.<br>
> >><br>
> >> Maybe it's biggest but at least there is only one copy of it:<br>
> >><br>
> >> top 10 in my downloads (archives and then git clones).<br>
> >><br>
> >> 823M<br>
> >> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.14.git.tar.gz<br>
> >> 831M<br>
> >> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.10.git.tar.gz<br>
> >> 893M<br>
> >> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.17.git.tar.gz<br>
> >> 934M<br>
> >> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.19.git.tar.gz<br>
> >> 969M<br>
> >> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-4.1.git.tar.gz<br>
> >> 1.2G<br>
> >> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-dev.git.tar.gz<br>
> >><br>
> >> 2.4G /OE/downloads/git2_github.com.gcc-mirror.gcc.tar.gz<br>
> >><br>
> >> 854M /OE/downloads/git2/github.com.qtproject.qtwebengine-chromium.git<br>
> >><br>
> >> 877M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.10.git<br>
> >> 900M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.14.git<br>
> >> 921M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.17.git<br>
> >> 964M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.19.git<br>
> >> 999M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-4.1.git<br>
> >> 1.3G /OE/downloads/git2/git.yoctoproject.org.linux-yocto-dev.git<br>
> >> 2.3G /OE/downloads/git2/git.yoctoproject.org.linux-yocto-4.4.git<br>
> ><br>
> > No argument from me there - it does seem a little wasteful given that<br>
> > there must be a huge overlap between those repos.<br>
><br>
> In other build systems, I've simply been able to reference a common<br>
> repository for the majority of the blobs. That isn't an option with<br>
> oe/bitbake (as far as I know).<br>
<br>
</div></div>It's possible we could add functionality along those lines to at least save<br>
download time; disk usage would probably be unaffected though.<br>
<span><br>
> We could also do shallow clones, but I'm also unsure of how well that<br>
> is supported.<br>
<br>
</span>I know Chris looked into adding this, I don't recall how far he got.</blockquote></div><div class="gmail_extra"><br></div>I did work on shallow support, yes.</div><div class="gmail_extra"><br>Shallow mirror tarball support is on the bitbake list awaiting Richard's review and merge for 2.2, assuming there are no major objections. There is a caveat, however. Since git's shallow clone capability is limited to specifying the depth, we can't really do shallow *clones*, as we don't know the distance from branch HEAD to SRCREV at the time we're going to clone.</div><div class="gmail_extra"><br></div><div class="gmail_extra">What we can support is shallow mirror tarballs, where a mirror is populated with shallow mirror tarballs which the user's build will fetch, if available, instead of full mirror tarballs, and this is what my patch series implements. Of course, each time the recipe SRCREVs change, we'd get new shallow tarballs that'd need fetching, enough of those would outweigh the benefits vs just doing a straight up front clone, but I think the benefit is clear. A shallow kernel tarball for a repo of this sort is around 100 megs instead of a gig.</div><div class="gmail_extra"><br></div><div class="gmail_extra">We've been using it at mentor for a while now with good results, so far. I'd love to see someone other than us test out the patches, though, or give some feedback on either the concept or the code :) See "[master][PATCH 0/3] Implement support for shallow git mirror tarballs" on the bitbake-devel list. <br>-- <br><div data-smartmail="gmail_signature"><div dir="ltr">Christopher Larson<br>kergoth at gmail dot com<br>Founder - BitBake, OpenEmbedded, OpenZaurus<br>Maintainer - Tslib<br>Senior Software Engineer, Mentor Graphics</div></div>
</div></div>