<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Apr 30, 2016 at 12:44 PM, Christopher Larson <span dir="ltr"><<a href="mailto:kergoth@gmail.com" target="_blank">kergoth@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">From: Christopher Larson <<a href="mailto:chris_larson@mentor.com">chris_larson@mentor.com</a>><br>
<br>
This adds support to the git fetcher for fetching and generating mirror<br>
tarballs of shallow git repositories, with the specified revisions / depths.<br>
<br>
This implements support for shallow mirror tarballs, not shallow clones.<br>
Supporting shallow clones directly would be rather more problematic, as we'd<br>
need to hardcode the depth between branch HEAD and the SRCREV, and that depth<br>
would change as the branch is updated. Sadly, git's remote protocol is<br>
extremely limited.<br>
<br>
When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow<br>
mirror tarball. If the shallow mirror tarball cannot be fetched, it will try<br>
to fetch the full mirror tarball and use that.<br>
<br>
Multiple variables exist to exert control over what revisions and refs are<br>
kept at what are not.<br>
<br>
BB_GIT_SHALLOW: enable/disable shallow support as a whole, boolean<br>
BB_GIT_SHALLOW_DEPTH: specify commit depth for the included refs. defaults to 1 (the top commit alone). This can be set to 0 or the empty string.<br>
BB_GIT_SHALLOW_DEPTH_<name>: specify commit depth for a specific named url/branch<br>
BB_GIT_SHALLOW_REVS: specific revisions whose history should be removed, beyond that specified by refs+depth<br>
BB_GIT_SHALLOW_TRIM_REFS: by default, unused refs/branches are removed. set this to 0 or the empty string to keep all refs around. this is required for linux-yocto kernel repositories, otherwise the branch checking done by the kernel scripts will fail<br>
BB_GIT_SHALLOW_EXTRA_REFS: additional refs to keep beyond those referenced in SRC_URI, when TRIM_REFS is enabled<br>
BB_GENERATE_SHALLOW_TARBALLS: explicitly control creation of shallow mirror tarballs. this defaults to enabled when BB_GENERATE_MIRROR_TARBALLS is enabled<br>
<br>
Example usage:<br>
<br>
BB_GIT_SHALLOW ?= "1"<br>
BB_GIT_SHALLOW_DEPTH ?= "1"<br>
<br>
# Usage in a recipe with multiple named uris or multiple named branches<br>
BB_GIT_SHALLOW_DEPTH_doc = ""<br>
<br>
# Remove the upstream history from our kernel repo, keeping our own<br>
BB_GIT_SHALLOW_REVS_pn-linux-mel_mx6 = "v3.14"<br>
BB_GIT_SHALLOW_DEPTH_pn-linux-mel_mx6 = ""<br>
<br>
# Keep the branches in linux-yocto repositories<br>
BB_GIT_SHALLOW_TRIM_REFS_pn-linux-yocto = "0"<br>
<br>
<br>
Please review the following changes for suitability for inclusion. If you have<br>
any objections or suggestions for improvement, please respond to the patches. If<br>
you agree with the changes, please provide your Acked-by.<br>
<br>
The following changes since commit 309f5907a3661821e041ed14645b5d165007b058:<br>
<br>
bitbake: Switch to post release version (2016-04-29 07:41:34 +0100)<br>
<br>
are available in the git repository at:<br>
<br>
<a href="https://github.com/kergoth/bitbake" rel="noreferrer" target="_blank">https://github.com/kergoth/bitbake</a> shallow-simplify<br>
<a href="https://github.com/kergoth/bitbake/tree/shallow-simplify" rel="noreferrer" target="_blank">https://github.com/kergoth/bitbake/tree/shallow-simplify</a><span class=""><font color="#888888"><br>
</font></span></blockquote></div><br><div>This branch has been updated after rebasing post-python3-switch, so should be ready for potential merge again. A couple things worth noting that I failed to mention in the original post:</div><div><br></div><div>The shallow mirror tarballs are never unpacked into DL_DIR -- I didn't want to create another dir structure there when there wouldn't be much benefit, and I definitely didn't want anything shallow inside of ${DL_DIR}/git2/ to avoid confusion and state problems, so when a shallow tarball is in play, the download() method will return early and do nothing beyond the fetch of the shallow tarball, and then the unpack method will extract it directly.</div><div><br></div><div>It's also worth noting that a current clone is preferred to either tarball, a shallow tarball is preferred to an out of date clone, and a missing clone will use either tarball.</div><div><br></div><div>I'd also like to clarify the usage of BB_GIT_SHALLOW_REVS. In a normal use of shallow, the history removed is that beyond a certain depth of a certain branch, so you end up with N commits left over, and everything else removed. If BB_GIT_SHALLOW_DEPTH isn't set and BB_GIT_SHALLOW_REVS is set, instead of only keeping a certain depth of history, we're only *removing* a particular subset of the history. The main use case is to get more control over what history is available to a customer when shipping sources, so e.g. you can strip out the bits available in public repositories and just keep the bits they can't easily re-fetch.</div><div><br></div><div>Richard, think you'll be able to find time to take a look at this? As long as BB_GIT_SHALLOW isn't set, there's almost no impact at all. The only change that isn't conditional upon this is the change in logic in bb.fetch to simplify the tarball handling, so the risk is actually quite low.</div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Christopher Larson<br>kergoth at gmail dot com<br>Founder - BitBake, OpenEmbedded, OpenZaurus<br>Maintainer - Tslib<br>Senior Software Engineer, Mentor Graphics</div></div>
</div></div>