<div><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le mar. 8 oct. 2019 à 22:54, Mark Hatle <<a href="mailto:mark.hatle@kernel.crashing.org">mark.hatle@kernel.crashing.org</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
On 10/8/19 1:45 PM, Aníbal Limón wrote:<br>
> In some cases people/organizations are using a SRC_URI with<br>
> file:///PATH_TO_DIR that contains a git repository for different<br>
> reasons, it is useful when want to do an internal build without<br>
> clone sources from outside.<br>
> <br>
> This could consume a lot of CPU time because the current taskhash<br>
> generation mechanism didn't identify that the folder is a VCS<br>
> (git, svn, cvs) and makes the cksum for every file including the<br>
> .git repository in this case.<br>
> <br>
> There are different ways to improve the situation,<br>
> <br>
> * Add protocol=gitscm in file:// SRC_URI but the taskhash is<br>
> calculated before the fetcher identifies the protocol, will require<br>
> some changes in bitbake codebase.<br>
<br>
When I have done this before, I've -always- defined it as a git repository by<br>
SRCURI:<br>
<br>
git://<local file path>;protocol=file<br>
<br>
Then it does exactly what this patch appears to do and uses the git logic to<br>
handle everything automatically. (The file protocol is already implemented.)<br>
<br>
Wouldn't this be better then specifying file:// and then attempting to infer<br>
what file refers to?</blockquote><div dir="auto"><br></div><div dir="auto">Anibal and I are working on the same use case / problem. Folks who rely on this setup also use it as a workspace, eg. They make local changes in the sources workspace. So when doing fetch again the local changes are used. So this is not strictly equivalent to what you propose, right?</div><div dir="auto"><br></div><div dir="auto">Another major flaw with the current file:// with a git folder is that doing a git fetch in the source tree will invalidate the fetch task even if the actual sources haven’t changed!</div><div dir="auto"><br></div><div dir="auto"><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
--MArk<br>
<br>
> * This patch: When directory is a git repository (contains .git)<br>
> use HEAD rev + git diff to calculate checksum instead of do it<br>
> in every file, that is hackish because make some assumptions about<br>
> .git directory contents.<br>
> * Variant of this patch: Make a list of VCS directories (.git, .svn,<br>
> .cvs) and take out for cksum calculations, same as before making<br>
> assumptions about the . folders content.<br>
> <br>
> Signed-off-by: Aníbal Limón <<a href="mailto:anibal.limon@linaro.org" target="_blank">anibal.limon@linaro.org</a>><br>
> ---<br>
> lib/bb/checksum.py | 13 +++++++++++++<br>
> 1 file changed, 13 insertions(+)<br>
> <br>
> diff --git a/lib/bb/checksum.py b/lib/bb/checksum.py<br>
> index 5bc8a8fc..ee125cb5 100644<br>
> --- a/lib/bb/checksum.py<br>
> +++ b/lib/bb/checksum.py<br>
> @@ -86,6 +86,19 @@ class FileChecksumCache(MultiProcessCache):<br>
> return checksum<br>
> <br>
> def checksum_dir(pth):<br>
> + git_dir = os.path.join(pth, '.git')<br>
> + if os.path.exists(git_dir):<br>
> + import subprocess, hashlib<br>
> + m = hashlib.md5()<br>
> + head = subprocess.check_output("cd %s && git rev-parse HEAD" % pth, shell=True)<br>
> + diff = subprocess.check_output("cd %s && git diff" % pth, shell=True)<br>
> + m.update(head)<br>
> + if diff:<br>
> + m.update(diff)<br>
> +<br>
> + return [(pth, m.hexdigest())]<br>
> +<br>
> +<br>
> # Handle directories recursively<br>
> if pth == "/":<br>
> bb.fatal("Refusing to checksum /")<br>
> <br>
-- <br>
_______________________________________________<br>
bitbake-devel mailing list<br>
<a href="mailto:bitbake-devel@lists.openembedded.org" target="_blank">bitbake-devel@lists.openembedded.org</a><br>
<a href="http://lists.openembedded.org/mailman/listinfo/bitbake-devel" rel="noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/bitbake-devel</a><br>
</blockquote></div></div>