[OE-core] How to find out why shared sstate is not being used for a recipe

[Mike Looijmans]

On 06/03/2014 10:45 AM, Richard Purdie wrote:
> On Tue, 2014-06-03 at 07:35 +0200, Mike Looijmans wrote:
>> On 06/03/2014 07:25 AM, Mike Looijmans wrote:
>>>> Worst case, you can pull the siginfo files from one build and the
>>>> siginfo files from the sstate mirror and then see which ones are
>>>> different, then run bitbake-diffsigs X Y to compare the two files.
>>>
>>> How do I find what to pull? I have (ssh) access to both machines. The
>>> sstate-cache dir contains a bunch of two-digit directories and a gazillion files.
>>>
>>> I could just copy the whole thing to one machine, there's gigabit between
>>> them, but then what do I do with these files?
>>
>> mike:~/zynq_platform/build$ find sstate-cache -name
>> '*fpga-image-miami*.siginfo' | wc -l
>> 480
>>
>> Suppose I copy them. Where do I copy them to, and what do I do with these 480
>> files to tell me why the system insists on rebuilding this package?
>>
>> Mike.
>>
>> PS: I really really want to find out. Several of these FPGA recipes take 3.5
>> hours to build on the fastest i5 we could buy. So you can imagine we really
>> want to prevent having to build it more than once.
>
> Let me explain the manual process you can follow for this. Its a pain to
> walk through and its what -S attempts to automate but you should be able
> to get an answer manually this way.
>
> You have BUILDA and BUILDB, the two builds which should be reusing
> sstate but are not. The fact they're on different machines is irrelevant
> to this procedure. It would help if these two builds are just the result
> of "bitbake fpga-image-miami" but that isn't essential, it will just
> introduce more noise. It will also help if they are either both built
> from an existing sstate cache or both not.
>
> The first thing I'd do in each build:
>
> "find tmp/stamps/ -type f | sort > stamplistA.txt"
>
> I'd then so something like:
>
> "meld stamplistA stamplistB"
>
> the next steps depend upon how clear the differences are. Basically
> there should be some degree of commonality between the two builds and at
> some point there will be divergence. We need to pinpoint the point of
> divergence. The divergence will be in fpga-image-miami itself or in one
> of its dependencies.
>
> The one thing which can confuse this view is if some things were reused
> from sstate. You can tell since a task which runs looks like:
>
> tmp/stamps/i586-poky-linux/x11-common/0.1-r47.do_populate_sysroot.d90d4404368125acd109a2ac64ca688f.qemux86
>
> and a task which is from sstate looks like:
>
> tmp/stamps/all-poky-linux/gstreamer1.0-meta-base/1.0-r0.do_populate_sysroot_setscene.a49fa811727c557c14ab6ce6f2973587.qemux86
>
> The "_setscene" part tells you this. Machine specific tasks have the
> machine name appended to them ".qemux86" in this case. The hash
> representing the task is clear in the filename
> ("a49fa811727c557c14ab6ce6f2973587" in this case").
>
> You'll have to filter out any "noise" from these changes you're not
> interested in. If a task is "_setscene" its dependencies may be missing
> from the list of files entirely (no install/compile/configure etc.).
>
> So you take a guess at the divergence point and take note of the two
> different hashes. You can then find the corresponding siginfo files in
> sstate:
>
> find sstate-cache/ -name *d90d4404368125acd109a2ac64ca688f*
> sstate-cache/d9/sstate:x11-common:i586-poky-linux:0.1:r47:i586:3:d90d4404368125acd109a2ac64ca688f_populate_sysroot.tgz
> sstate-cache/d9/sstate:x11-common:i586-poky-linux:0.1:r47:i586:3:d90d4404368125acd109a2ac64ca688f_populate_sysroot.tgz.siginfo
>
> which are the sstate files corresponding to my above x11-common task.
> You will note that the first two letters of the hash are used as a
> directory prefix. You can also find sigdata files in the stamp
> directory:
>
> $ find tmp/stamps/ -type f | grep d90d440
> tmp/stamps/i586-poky-linux/x11-common/0.1-r47.do_populate_sysroot.d90d4404368125acd109a2ac64ca688f.qemux86
> tmp/stamps/i586-poky-linux/x11-common/0.1-r47.do_populate_sysroot.sigdata.d90d4404368125acd109a2ac64ca688f
>
> The sigdata and siginfo files are identical and equivalent. Once you
> have the two sstate files, you can run:
>
> bitbake-diffsigs A.siginfo B.siginfo
>
> and this will tell you why their hashes are different. If you need help
> deciphering that output, link to it and I can help.

Dependency on task fpga-image-medical-demo.bb.do_patch was added with hash 
8fbb54a572a402e28d05daae0a238f8b
Dependency on task fpga-image-medical-demo.bb.do_patch was removed with hash 
44a252a24b20321e8dbec6d9d8b54c67

Now this means that I have to look at the "unpack" task now, right?

The weird thing is, both machines have identical do_patch hashes:
tmp-eglibc/stamps/zynq_miami_7030-oe-linux-gnueabi/fpga-image-medical-demo/0.AUTOINC-06349d0386-r0.do_patch.sigdata.44a252a24b20321e8dbec6d9d8b54c67

I cannot find that "8fbb54a572a402e28d05daae0a238f8b" hash anywhere.





Met vriendelijke groet / kind regards,

Mike Looijmans

TOPIC Embedded Systems
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: (+31) (0) 499 33 69 79
Telefax:  (+31) (0) 499 33 69 70
E-mail: mike.looijmans at topic.nl
Website: www.topic.nl

Please consider the environment before printing this e-mail

Visit us at MEDTEC Europe 3-5 June, Messe Stuttgart, stand number 5D20
http://medteceurope.com/index.php?page=home-en