[OE-core] [PATCH] [RFC] go.bbclass: install dependencies under $GOPATH instead of $GOROOT

[Damien Riegel]

On Wed, 19 Dec 2018 at 16:41, Matt Madison <matt at madison.systems> wrote:
>
> On Wed, Dec 19, 2018 at 9:51 AM Damien Riegel <damien.riegel at gmail.com> wrote:
> >
> > To package a go application in Yocto, one needs to also package its
> > dependencies to keep the build reproducible and under control. The
> > default install task of the go class puts the source in ${libdir}/go/src.
> >
> > ${libdir}/go/src is where the standard go packages resides, aka $GOROOT.
> > When a go package has dependencies on other go packages, they must be
> > located under $GOPATH, otherwise errors like this one may happen:
> >
> >   Non-standard package <packagename> in standard package <packagename>
> >
> > This point of this patch is to trigger a discussion on how this issue
> > can be tackled in Yocto. Following code is working but should not be
> > considered for inclusion.
>
> Can you describe the context of this a bit more?  The way go.bbclass
> is set up, all of your recipe's dependencies (and their dependencies)
> should already have been installed in the per-recipe sysroot (where
> GOROOT lives for the current recipe's build), and should get resolved
> from there. GOPATH gets set to ${B} for the current recipe's build.
> Nothing in the GOROOT should be importing packages that reside in
> GOPATH, which is what that particular error message is indicating. If
> you're getting that error, then there might be something wrong with
> recipe dependencies.

Sure. I think the issue is because I have the following chain of
dependencies:

A --> B --> C
  `-> C

Building C is fine, building B is fine, but building A triggers the
error I mentioned, because the Go compiler complains about B using a
non-standard package C. It thinks B is a standard package because it's
located in GOROOT.

Regarding dependencies, I created recipes named foocorp.com-{A,B,C}.bb
and used DEPENDS_append = " foocorp.com-A" for instance.

I understand it's convenient to install dependencies in the GOROOT
directory of recipes' sysroot, but we should keep in mind that when one
calls `go get`, dependencies are installed in GOPATH, not GOROOT. I
think that if we want to avoid (a bit obscure) build failures and cover
all use cases, we should try to mimic that behaviour as close as we can.

>
> -Matt
> >
> > Signed-off-by: Damien Riegel <damien.riegel at gmail.com>
> > ---
> >  meta/classes/go.bbclass | 18 +++++++++++++++---
> >  1 file changed, 15 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> > index 167d02e3fa..c0ed3211cd 100644
> > --- a/meta/classes/go.bbclass
> > +++ b/meta/classes/go.bbclass
> > @@ -111,9 +111,9 @@ do_compile_ptest_base() {
> >  do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
> >
> >  go_do_install() {
> > -       install -d ${D}${libdir}/go/src/${GO_IMPORT}
> > +       install -d ${D}/gopath/src/${GO_IMPORT} ${D}${libdir}/go
> >         tar -C ${S}/src/${GO_IMPORT} -cf - --exclude-vcs --exclude '*.test' --exclude 'testdata' . | \
> > -               tar -C ${D}${libdir}/go/src/${GO_IMPORT} --no-same-owner -xf -
> > +               tar -C ${D}/gopath/src/${GO_IMPORT} --no-same-owner -xf -
> >         tar -C ${B} -cf - pkg | tar -C ${D}${libdir}/go --no-same-owner -xf -
> >
> >         if [ -n "`ls ${B}/${GO_BUILD_BINDIR}/`" ]; then
> > @@ -172,9 +172,21 @@ do_install_ptest_base() {
> >         chown -R root:root ${D}${PTEST_PATH}
> >  }
> >
> > +do_fixupdeps() {
> > +       gopath="${WORKDIR}/recipe-sysroot/gopath"
> > +       if [ -d ${gopath} ]; then
> > +               tar -C ${gopath} -cf - src | tar -C ${B} --no-same-owner -k -xf -
> > +       else
> > +               echo "no dependencies to fixup"
> > +       fi
> > +}
> > +
> >  EXPORT_FUNCTIONS do_unpack do_configure do_compile do_install
> >
> > -FILES_${PN}-dev = "${libdir}/go/src"
> > +addtask do_fixupdeps after do_configure before do_compile
> > +
> > +SYSROOT_DIRS_append = "/gopath"
> > +FILES_${PN}-dev = "/gopath/src"
> >  FILES_${PN}-staticdev = "${libdir}/go/pkg"
> >
> >  INSANE_SKIP_${PN} += "ldflags"
> > --
> > 2.19.2
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core at lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core