[OE-core] [PATCH] metadata-revs: provide more information

[Trevor Woerner]

(sorry about my previous reply, somehow the quoting got messed up and 
was hard to read. hopefully this one is better)

Hi Paul,

On 03/13/16 15:26, Paul Eggleton wrote:
> On Sat, 12 Mar 2016 21:35:29 Trevor Woerner wrote:
>> Provide many more details concerning the repositories that are used in a
>> particular build: the remote information, the layer, the local branch, the
>> remote branch the local branch tracks (if any), and the HEAD commit.
>>
>> Signed-off-by: Trevor Woerner <twoerner at gmail.com>
>> ---
>>   meta/classes/buildhistory.bbclass |  6 +++++-
>>   meta/classes/metadata_scm.bbclass | 18 ++++++++++++++++++
>>   2 files changed, 23 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/buildhistory.bbclass
>> b/meta/classes/buildhistory.bbclass index b6b4324..1b0ae0e 100644
>> --- a/meta/classes/buildhistory.bbclass
>> +++ b/meta/classes/buildhistory.bbclass
>> @@ -615,9 +615,13 @@ def buildhistory_get_build_id(d):
>>
>>   def buildhistory_get_metadata_revs(d):
>>       # We want an easily machine-readable format here, so
>> get_layers_branch_rev isn't quite what we want +    import subprocess
>>       layers = (d.getVar("BBLAYERS", True) or "").split()
>> -    medadata_revs = ["%-17s = %s:%s" % (os.path.relpath(i,
>> d.getVar('BBLAYERS_FETCH_DIR', True)), \ +    medadata_revs = ["%s\tlayer:
>> %s\n\tbranch: %s\n\tremote: %s\n\tHEAD:   %s\n" % ( \ +
>> base_get_metadata_git_remote(i, None), \
>> +        os.path.relpath(i, d.getVar('BBLAYERS_FETCH_DIR', True)), \
>>           base_get_metadata_git_branch(i, None).strip(), \
>> +        base_get_metadata_git_remote_branch(i, None).strip(), \
>>           base_get_metadata_git_revision(i, None)) \
>>               for i in layers]
>>       return '\n'.join(medadata_revs)
>> diff --git a/meta/classes/metadata_scm.bbclass
>> b/meta/classes/metadata_scm.bbclass index 0f7f423..31a2c54 100644
>> --- a/meta/classes/metadata_scm.bbclass
>> +++ b/meta/classes/metadata_scm.bbclass
>> @@ -73,6 +73,15 @@ def base_get_metadata_git_branch(path, d):
>>           rev = '<unknown>'
>>       return rev.strip()
>>
>> +def base_get_metadata_git_remote_branch(path, d):
>> +    import bb.process
>> +
>> +    try:
>> +        rev, _ = bb.process.run('git rev-parse --abbrev-ref
>> --symbolic-full-name @{u}', cwd=path) +    except
>> bb.process.ExecutionError:
>> +        rev = '(HEAD does not point to a remote branch)'
>> +    return rev.strip()
>> +
>>   def base_get_metadata_git_revision(path, d):
>>       import bb.process
>>
>> @@ -81,3 +90,12 @@ def base_get_metadata_git_revision(path, d):
>>       except bb.process.ExecutionError:
>>           rev = '<unknown>'
>>       return rev.strip()
>> +
>> +def base_get_metadata_git_remote(path, d):
>> +    import bb.process
>> +
>> +    try:
>> +        lines, _ = bb.process.run('git remote -v', cwd=path)
>> +    except bb.process.ExecutionError:
>> +        return '<unknown>'
>> +    return lines
>
> As I mentioned in my other reply, metadata-revs was intended to be consumed by
> scripts rather than humans, so I'd rather not change its format unless
> absolutely necessary.


No problem, I had guessed that might be the case.

Would you (or anyone) be open to the possibility of the buildhistory 
task creating a second file that would contain this information?

I had been thinking about doing something like this for a long time. 
Having the two "meta" listings is what finally prompted me to actually 
do something about it. The fact that Ross pointed out that it was really 
the result of my own mistake is a little bit secondary to what I'm 
trying to accomplish.

I like to keep my build artifacts for a very long time. But I find when 
I'm looking through my old artifacts that I wish I had also kept the 
various configurations that led to that artifact's creation. ["Wait... I 
successfully built something for MACHINE XYZ back in June, how did I do 
that?"] So, along with the artifacts, I've also been keeping local.conf, 
auto.conf, bblayers.conf... and metadata-revs.

However, as I've pointed out, there are lots of clones around. Have you
taken a look at github recently? There are dozens of clones + tweaks to
many of the repositories we all know and love: beaglebone, odroid,
raspberrypi, browser. Sometimes it's one of these clones that works out
better for a given situation (MACHINE, board...) than the canonical. For 
example, if I want to add chromium to my DragonBoard410c build, I need 
to use Linaro's clone of meta-browser and not the canonical OSSystems' one.

The information currently provided by buildhistory/metadata-revs doesn't 
provide the level of detail required to distinguish between these two 
situations. In six months I would look at what I've built today, grab 
git://github.com/OSSystems/meta-browser.git, and then say: "how come 
there's no commit 5c00d0114c5963a178cb33f6d06181c588c03ae0?". It's not 
like there's any requirement to make the names of all layers universally 
unique.

That's the problem I'm trying to solve: how can I easily keep all the
information required to reproduce this build, exactly. The whole "two
meta's" thing was just a speed bump.