[bitbake-devel] git fetcher and github pull requests

[Nicolas Dechesne]

hi there,

we've been debugging an issue these days on our builder which ended up
 do_fetch failing with

LANG=C git -c core.fsyncobjectfiles=0 fetch -f --prune --progress
git://github.com/OP-TEE/optee_test.git refs/*:refs/* failed with exit
code 128, output:
error: Could not read 48e440f5f8d033e1ace6e41f424ecf6e6d96e5f2
error: Could not read 019a8db54beb29388e1108831d2e2dc135c1cd73

It happens that these refs correspond to pull requests done on github
which existed at some point, but have been updated with newer commits,
and won't exist anymore.

the bitbake fetcher seems to be greedy and fetches refs/* which ends
up fetching pull request when fetching from github, e.g. in my
workspace:

in $DL_DIR/git2/

github.com.docker.containerd.git/refs/pull/66
github.com.docker.containerd.git/refs/pull/459
github.com.docker.containerd.git/refs/pull/551
github.com.docker.containerd.git/refs/pull/321
github.com.docker.containerd.git/refs/pull/60
github.com.docker.containerd.git/refs/pull/523
github.com.docker.containerd.git/refs/pull/40
github.com.docker.containerd.git/refs/pull/561

It looks inefficient to fetch and store on each builder pull requests.
I understand this is just because how PR are implemented in github,
but github is quite central, so many we should/could do something
about it?

Beyond the inefficiencies, we now are seeing unrelated build issues as well.

What do you think? Should we try to avoid fetching refs/pull/* from
github? or is it our git fetch command that needs to be improve to
work in this situation?

thanks
nico