[OE-core] Overriding fixmepath

[Kristian Amlie]

On 25/01/17 12:24, Richard Purdie wrote:
> On Wed, 2017-01-25 at 11:43 +0100, Kristian Amlie wrote:
>> After the new recipe sysroots were introduced we've been having
>> trouble
>> compiling Go. I've narrowed it down to the path replacement mechanism
>> that goes on using fixmepath.
>>
>> For example, this does not work:
>>
>>   bitbake -c cleansstate go-cross
>>   bitbake go-cross
>>
>> But this does:
>>
>>   bitbake -c cleansstate go-cross
>>   bitbake -c prepare_recipe_sysroot go-cross
>>   rm -rf
>> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-
>> linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-
>> native-1.4.3
>>   cp -r
>> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
>> components/x86_64/go-bootstrap-native/usr/lib/go-bootstrap-native-
>> 1.4.3
>> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/work/x86_64-
>> linux/go-cross/1.7.4-r0/recipe-sysroot-native/usr/lib/go-bootstrap-
>> native-1.4.3
>>   bitbake go-cross
>>
>> IOW, if I replace the recipe sysroot with the originals, it works,
>> but
>> if I keep the modified files put there by the prepare_recipe_sysroot
>> task, it doesn't.
>>
>> And this is where I admittedly get a bit lost: I haven't fully
>> understood how the fixmepath mechanism operates, but from what I can
>> tell, this path replacement is not appropriate for Go. It doesn't
>> care
>> where it's installed, and always operates relative to the GOROOT
>> environment variable, which is set by all Go recipes.
> 
> Does this still work if you remove the
> /home/jenkins/workspace/yoctobuild/build-qemu/tmp/sysroots-
> components/x86_64/go-bootstrap-native directory before building go-
> cross? I worry that its actually referencing the other location and
> using files from there (a different recipe's workdir) and that
> therefore there are a different set of other problems this recipe also
> has.

You mean after executing prepare_recipe_sysroot, right? I tried this,
but it made no difference to my previous results, so presumably it is
looking in the right place.

> 
>> Why the path replacement triggers an error I cannot say, but I
>> suspect it has to do with the fact that Go object files are a bit
>> special compared to C object files, and this operation in fact
>> corrupts them. More mysterious still is why this happens only on some
>> build machines, and not others, but once triggered it appears to be
>> consistently triggered.
> 
> The fixmepath replacements only happen on text files. It could be some
> relocation is needed on the binaries too, perhaps through configuration
> on the commandline or perhaps through the environment. We have had to
> patch some tools to allow relocation in the past too.

I see this happening to .a files as well. Aren't they normally object
files? I have attached the fixmepath file for go-bootstrap-native, which
go-cross uses to build itself.

Looking at one of the files from there, it has quite peculiar contents,
probably something Go internal which we shouldn't mess with:

  $ ar t template.a
  __.PKGDEF
  _go_.6

>> Any advice for what to do next? I could try to skip the fixmepath
>> operation completely, but I'm not sure how, and it feels a bit like
>> using a sledgehammer where I should be using a scalpel.
> 
> Try removing the path I suggested after copying and see if it still
> works. If it does, I'd probably try and find which subset of the files
> breaks it, narrow down the issue. If removing it breaks things, there
> is a whole world of bigger issues :(
> 
> Its possible to add specific inclusions/exclusions from fixmepath FWIW
> but we need to understand more about what is happening first.

Take a look at the attached file, to me the inclusion of .a files in the
fixmepath file seems wrong, so I can start with that, unless you have
better idea.

-- 
Kristian
-------------- next part --------------
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/html/template.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/syscall.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/crypto/x509.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/crypto/tls.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/text/template.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/go/ast.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/debug/elf.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/cgi.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/httputil.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/fcgi.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/httptest.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/http/pprof.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/smtp.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net/rpc.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/net.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/cmd/pprof/internal/driver.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/pkg/linux_amd64/cmd/internal/rsc.io/arm/armasm.a
recipe-sysroot-native/usr/lib/go-bootstrap-native-1.4.3/src/runtime/zversion.go