[oe] Hash based SCMs and OpenEmbedded/Bitbake

[Holger Freyther]

Hey guys,

we have a couple of issues with hash based SCMs (like git) and how we handle 
them.

1.) sane-srcrev. During stabilisation you want to lock down a rev, so you set 
one in sane-srcrev.

Issue: The PV says something like PV = "1.0.0+gitr${SRCREV}". Now the SRCREV 
is fixed and will not go through the python hook. So the resulting package 
will be PV = "1.0.0+gitrdeadbeef" and if you want to upgrade to the 
ref "affe" you will have a package called "1.0.0+gitraffe" which is not 
upgradable.

Approach at Openmoko: Invent "SRCPV" and put that into the PV line. SRCPV will 
go throuh the bitbake fetcher hook and will generate a string which includes 
an increasing number (like on the autorev's).

This will allow people to set a SRCREV and people will be able to upgrade. 
Afterward you can remove PR from the PV line to allow such things...

This involves a change to conf/bitbake.conf and a fix to the various bb files 
to use SRCPV (which was also used to fix the PV line to say +SCMr..)


2.) The monotonely increasing number is completely random and local. So if two 
people generate packages on two different systems the one having built more 
often (more revs) will have the upgradable package.

Approach at Openmoko: We assume that upstream has a fast forward history (no 
rev is suddenly vanishing) and we will use git-rev-list to count the revs 
(from henryk) to get a monotonic revision number. This means suddenly the 
monotonic number is global. This patch is optional and needs to be enabled 
with a config key.

Issues:
	- Assume history is fast forward, if not packages are not upgraded
	- This forces to have a local git tree (making distributing git-checkouts 
impossible) of everything bitbake will look at during parse.

Werner Almesberger's proposal:
	- For the decision if we should rebuild or not any id is good enough. So we 
could ping/check upstream, check if the last rev was different and then 
schedule it for rebuild (like we do now)
	- When packaging we will use the right revision and put that into the 
package.

Issues:
	+ Only git repos of what we need! Faster parse time
	- tmp/work/foo-PV-PR and tmp/deploy/glibc/ipk/... will not relate..

 

comments, I would be happy to merge some of those bits from Openmoko to 
Openembedded.


	z.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Allow-to-conditionally-implement-sortable-revision-i.patch
Type: text/x-diff
Size: 1121 bytes
Desc: not available
URL: <http://lists.openembedded.org/pipermail/openembedded-devel/attachments/20080903/ec40acd0/attachment-0006.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-git-Optionally-use-git-rev-list-to-get-a-sortable.patch
Type: text/x-diff
Size: 2213 bytes
Desc: not available
URL: <http://lists.openembedded.org/pipermail/openembedded-devel/attachments/20080903/ec40acd0/attachment-0007.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0007-git-Do-not-run-git-rev-list-everytime-to-increase.patch
Type: text/x-diff
Size: 1392 bytes
Desc: not available
URL: <http://lists.openembedded.org/pipermail/openembedded-devel/attachments/20080903/ec40acd0/attachment-0008.bin>