[bitbake-devel] [PATCH] fetch2/git: always use premirror first if update is required

[Pascal Bach]

On 19.09.2016 17:10, Christopher Larson wrote:
>
> On Mon, Sep 19, 2016 at 12:22 AM, Pascal Bach <pascal.bach at siemens.com <mailto:pascal.bach at siemens.com>> wrote:
>
>     On 16.09.2016 18:20, Christopher Larson wrote:
>
>     >
>     > On Fri, Sep 16, 2016 at 1:58 AM, Pascal Bach <pascal.bach at siemens.com <mailto:pascal.bach at siemens.com> <mailto:pascal.bach at siemens.com <mailto:pascal.bach at siemens.com>>> wrote:
>     >
>     >     When an update of the local copy is required always try to fetch the update
>     >     from a premirror first.
>     >     The premirror should always have precedence over the upstream repository
>     >     as updating from there might not work if BB_FETCH_PREMIRRORONLY,
>     >     BB_NO_NBB_ALLOWED_NETWORKS or BB_NO_NETWORK is set.
>     >
>     >     This also gets rid of the special case for BB_FETCH_PREMIRRORONLY.
>     >
>     >     Signed-off-by: Pascal Bach <pascal.bach at siemens.com <mailto:pascal.bach at siemens.com> <mailto:pascal.bach at siemens.com <mailto:pascal.bach at siemens.com>>>
>     >     ---
>     >      lib/bb/fetch2/git.py | 9 ++-------
>     >      1 file changed, 2 insertions(+), 7 deletions(-)
>     >
>     >     diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
>     >     index 1bec60a..cac2a87 100644
>     >     --- a/lib/bb/fetch2/git.py
>     >     +++ b/lib/bb/fetch2/git.py
>     >     @@ -227,13 +227,8 @@ class Git(FetchMethod):
>     >              return False
>     >
>     >          def try_premirror(self, ud, d):
>     >     -        # If we don't do this, updating an existing checkout with only premirrors
>     >     -        # is not possible
>     >     -        if d.getVar("BB_FETCH_PREMIRRORONLY", True) is not None:
>     >     -            return True
>     >     -        if os.path.exists(ud.clonedir):
>     >     -            return False
>     >     -        return True
>     >     +        # Always try premirror if an update is required
>     >     +        return self.need_update(ud, d)
>     >
>     >
>     > Will this result in fetching a mirror tarball even if you already have a clone in ud.clonedir? The previous logic always returned False if the clonedir existed..
>     The tarball should only be fetched if the local ud.clonedir is out of date, this means the revision that is requested via SRCREV is not in the local repository.
>     In the case the revision is present the tarball should not be fetched again, at least that was the intention and my local tests didn't indicate otherwise.
>
>
> Yes, but it’s better to run ‘git fetch’ in an out of date clone than to re-download a 1gb mirror tarball which may well be out of date too.
Agreed, but this doesn't work in the case where the machine doesn't have access to the upstream git repository. For example in the case where BB_ALLOWED_NETWORKS = "*.my.company". The fetcher would still got to to github.com which is not allowed, in that case it should fall back to fetch the tarball from "mirror.my.company".

I'm open for suggestions how to make this more efficient.

Pascal