<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
</head>
<body>
<div name="messageBodySection">
<div dir="auto">Don’t have time for a full reply at the moment, I’ll take a look in the morning, but I did want to note that git2_boringssl.googlesource.com.boringssl.tar.gz is a full git mirror tarball as generated by BB_GENERATE_MIRROR_TARBALLS, and the premirrors fetching will download and use them if you want to skip the git-http-backend approach entirely. They are *not* shallow in any way. Only gitshallow_*.tar.gz are shallow tarballs.</div>
</div>
<div name="messageReplySection">On Oct 25, 2019, 1:39 PM -0700, Konrad Scherer <Konrad.Scherer@windriver.com>, wrote:<br />
<blockquote type="cite" class="spark_quote" style="margin: 5px 5px; padding-left: 10px; border-left: thin solid #1abc9c;">Hello,<br />
<br />
I recently found a strange problem involving the gitsm fetcher and our PREMIRRORS and I was able to<br />
reproduce it using poky.<br />
<br />
<blockquote type="cite" class="spark_quote" style="margin: 5px 5px; padding-left: 10px; border-left: thin solid #e67e22;">git clone git://git.yoctoproject.org/poky poky1<br />
cd poky1<br />
. oe-init-build-env<br />
bitbake ovmf<br /></blockquote>
<br />
The ovmf recipe has git submodules inside git submodules. This step works fine.<br />
<br />
I then exposed the downloads directory over http (http://<host>/downloads) and the downloads/git2<br />
directory over http with git-http-backend enabled (http://<host>/git).<br />
<br />
I noticed that there are shallow clone tarballs in downloads that match the git repos in<br />
downloads/git2. For example:<br />
<br />
downloads/git2/boringssl.googlesource.com.boringssl/<br />
downloads/git2_boringssl.googlesource.com.boringssl.tar.gz<br />
<br />
Since these are duplicates of the git repos, I deleted the tar.gz files<br />
<br />
<blockquote type="cite" class="spark_quote" style="margin: 5px 5px; padding-left: 10px; border-left: thin solid #e67e22;">rm -f downloads/git2_*<br /></blockquote>
<br />
<new shell><br />
<br />
<blockquote type="cite" class="spark_quote" style="margin: 5px 5px; padding-left: 10px; border-left: thin solid #e67e22;">git clone git://git.yoctoproject.org/poky poky2<br />
cd poky2<br />
. oe-init-build-env<br />
cat >> conf/local.conf <<EOF<br /></blockquote>
WRS_MIRROR_HOST = "<host>"<br />
BB_ALLOWED_NETWORKS = "${WRS_MIRROR_HOST}"<br />
<br />
BB_NO_NETWORK = '0'<br />
BB_FETCH_PREMIRRORONLY = '1'<br />
<br />
PREMIRRORS_append = " \<br />
.*://.*/.* http://${WRS_MIRROR_HOST}/downloads/ \n \<br />
git://.*/.* git://${WRS_MIRROR_HOST}/git/MIRRORNAME;protocol=http \n \<br />
gitsm://.*/.* git://${WRS_MIRROR_HOST}/git/MIRRORNAME;protocol=http \n \<br />
"<br />
CONNECTIVITY_CHECK_URIS = ""<br />
EOF<br />
<blockquote type="cite" class="spark_quote" style="margin: 5px 5px; padding-left: 10px; border-left: thin solid #e67e22;">bitbake ovmf<br /></blockquote>
<br />
This fails with:<br />
<br />
ERROR: ovmf-edk2-stable201905-r0 do_unpack: gitsm: submodule unpack failed: UnpackError Unpack<br />
failure for URL:<br />
'gitsm://github.com/openssl/openssl;protocol=https;name=CryptoPkg/Library/OpensslLib/openssl;subpath=CryptoPkg/Library/OpensslLib/openssl;bareclone=1;nobranch=1'.<br />
No up to date source found: clone directory not available or not up to date:<br />
/ala-lpggp22/kscherer/gitsm/poky2/build/downloads/git2/github.com.openssl.openssl; shallow clone not<br />
enabled<br />
<br />
If I leave the shallow clone tarballs in the PREMIRROR the build succeeds. The other recipes that<br />
use git are fetched properly. The logs don't have any warnings before the error and I verified that<br />
the objects are indeed present in the repo.<br />
<br />
Are my PREMIRROR settings correct? Is this a bug in the gitsm fetcher?<br />
<br />
Any help and insight appreciated.<br />
<br />
--<br />
Konrad Scherer, MTS, Linux Products Group, Wind River<br />
--<br />
_______________________________________________<br />
bitbake-devel mailing list<br />
bitbake-devel@lists.openembedded.org<br />
http://lists.openembedded.org/mailman/listinfo/bitbake-devel<br /></blockquote>
</div>
</body>
</html>