[OE-core] [PATCH] [RFC] go.bbclass: install dependencies under $GOPATH instead of $GOROOT

[Matt Madison]

On Wed, Dec 19, 2018 at 5:52 PM Damien Riegel <damien.riegel at gmail.com> wrote:
>
> On Wed, 19 Dec 2018 at 16:41, Matt Madison <matt at madison.systems> wrote:
> >
> > On Wed, Dec 19, 2018 at 9:51 AM Damien Riegel <damien.riegel at gmail.com> wrote:
> > >
> > > To package a go application in Yocto, one needs to also package its
> > > dependencies to keep the build reproducible and under control. The
> > > default install task of the go class puts the source in ${libdir}/go/src.
> > >
> > > ${libdir}/go/src is where the standard go packages resides, aka $GOROOT.
> > > When a go package has dependencies on other go packages, they must be
> > > located under $GOPATH, otherwise errors like this one may happen:
> > >
> > >   Non-standard package <packagename> in standard package <packagename>
> > >
> > > This point of this patch is to trigger a discussion on how this issue
> > > can be tackled in Yocto. Following code is working but should not be
> > > considered for inclusion.
> >
> > Can you describe the context of this a bit more?  The way go.bbclass
> > is set up, all of your recipe's dependencies (and their dependencies)
> > should already have been installed in the per-recipe sysroot (where
> > GOROOT lives for the current recipe's build), and should get resolved
> > from there. GOPATH gets set to ${B} for the current recipe's build.
> > Nothing in the GOROOT should be importing packages that reside in
> > GOPATH, which is what that particular error message is indicating. If
> > you're getting that error, then there might be something wrong with
> > recipe dependencies.
>
> Sure. I think the issue is because I have the following chain of
> dependencies:
>
> A --> B --> C
>   `-> C
>
> Building C is fine, building B is fine, but building A triggers the
> error I mentioned, because the Go compiler complains about B using a
> non-standard package C. It thinks B is a standard package because it's
> located in GOROOT.

Are the three packages in separate source repositories? If so, the go
tool should not be finding any C files in GOPATH.

If A and C are part of the same source repository, but B is separate,
that would make the situation a bit trickier, but still solvable.

> Regarding dependencies, I created recipes named foocorp.com-{A,B,C}.bb
> and used DEPENDS_append = " foocorp.com-A" for instance.

If A imports from B and C, you should have DEPENDS = "foocorp.com-B
foocorp.com-C" in the recipe for A, and
DEPENDS = "foocorp.com-C" in the recipe for B.

> I understand it's convenient to install dependencies in the GOROOT
> directory of recipes' sysroot, but we should keep in mind that when one
> calls `go get`, dependencies are installed in GOPATH, not GOROOT. I
> think that if we want to avoid (a bit obscure) build failures and cover
> all use cases, we should try to mimic that behaviour as close as we can.

go.bbclass does not use `go get`.  It only uses `go install`.

-Matt

>
> >
> > -Matt
> > >
> > > Signed-off-by: Damien Riegel <damien.riegel at gmail.com>
> > > ---
> > >  meta/classes/go.bbclass | 18 +++++++++++++++---
> > >  1 file changed, 15 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> > > index 167d02e3fa..c0ed3211cd 100644
> > > --- a/meta/classes/go.bbclass
> > > +++ b/meta/classes/go.bbclass
> > > @@ -111,9 +111,9 @@ do_compile_ptest_base() {
> > >  do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
> > >
> > >  go_do_install() {
> > > -       install -d ${D}${libdir}/go/src/${GO_IMPORT}
> > > +       install -d ${D}/gopath/src/${GO_IMPORT} ${D}${libdir}/go
> > >         tar -C ${S}/src/${GO_IMPORT} -cf - --exclude-vcs --exclude '*.test' --exclude 'testdata' . | \
> > > -               tar -C ${D}${libdir}/go/src/${GO_IMPORT} --no-same-owner -xf -
> > > +               tar -C ${D}/gopath/src/${GO_IMPORT} --no-same-owner -xf -
> > >         tar -C ${B} -cf - pkg | tar -C ${D}${libdir}/go --no-same-owner -xf -
> > >
> > >         if [ -n "`ls ${B}/${GO_BUILD_BINDIR}/`" ]; then
> > > @@ -172,9 +172,21 @@ do_install_ptest_base() {
> > >         chown -R root:root ${D}${PTEST_PATH}
> > >  }
> > >
> > > +do_fixupdeps() {
> > > +       gopath="${WORKDIR}/recipe-sysroot/gopath"
> > > +       if [ -d ${gopath} ]; then
> > > +               tar -C ${gopath} -cf - src | tar -C ${B} --no-same-owner -k -xf -
> > > +       else
> > > +               echo "no dependencies to fixup"
> > > +       fi
> > > +}
> > > +
> > >  EXPORT_FUNCTIONS do_unpack do_configure do_compile do_install
> > >
> > > -FILES_${PN}-dev = "${libdir}/go/src"
> > > +addtask do_fixupdeps after do_configure before do_compile
> > > +
> > > +SYSROOT_DIRS_append = "/gopath"
> > > +FILES_${PN}-dev = "/gopath/src"
> > >  FILES_${PN}-staticdev = "${libdir}/go/pkg"
> > >
> > >  INSANE_SKIP_${PN} += "ldflags"
> > > --
> > > 2.19.2
> > >
> > > --
> > > _______________________________________________
> > > Openembedded-core mailing list
> > > Openembedded-core at lists.openembedded.org
> > > http://lists.openembedded.org/mailman/listinfo/openembedded-core