[oe] PRINC migration questions

[Martin Jansa]

On Fri, Nov 07, 2014 at 03:36:14PM +0000, Bryan Evenson wrote:
> Martin,
> 
> > -----Original Message-----
> > From: openembedded-devel-bounces at lists.openembedded.org
> > [mailto:openembedded-devel-bounces at lists.openembedded.org] On
> > Behalf Of Martin Jansa
> > Sent: Friday, November 07, 2014 9:36 AM
> > To: openembedded-devel at lists.openembedded.org
> > Subject: Re: [oe] PRINC migration questions
> > 
> > On Fri, Nov 07, 2014 at 01:31:02PM +0000, Bryan Evenson wrote:
> > > All,
> > >
> > > I am on poky/dylan and have not yet started using the PR server.  I really do
> > want to start using the PR server and stop using PRINC.  However, even more
> > so I really don't want to break my package feeds.  I want to make sure I do
> > my migration correctly and I don't do something that looks like it works okay
> > only to find out it causes problems later.  With that in mind, I am looking for
> > advice on the proper way to migrate away from using PRINC.
> > >
> > > Let's say I have a .bbappend in my private layer with the line:
> > >
> > > PRINC := "${@int(PRINC) + 4}"
> > >
> > > The mainline layer's .bb has the line:
> > >
> > > PR = "${INC_PR}.0"
> > >
> > > And the mainline layer's .inc has the line:
> > >
> > > INC_PR = "r8"
> > >
> > > In this scenario, the resulting PR will be "r12.0".  Now let's say I want to start
> > using the PR server and get rid of PRINC in this recipe.  The resulting package
> > contents are to be the same, so I want the resulting PR to be "r12.0".  From
> > some limited testing, if I start the PR server and remove the PRINC line from
> > my .bbappend the resulting PR is "r8.0".  So in my .bbappend should I change
> > the PRINC line to:
> > >
> > > INC_PR = "r12"
> > >
> > > Or should I change it to something else?  If I change it to a hardcoded value,
> > then I'll have to be careful about what to do if the mainline recipe changes
> > INC_PR.  How have other people handled this situation?
> > 
> > PR service won't you help at all with this (it adds another .N to PR value, so it
> > cannot fix when the level of .N goes backwards).
> > 
> > The only really working solution is to increment INC_PR/PR values in
> > upstream recipes atomically with PRINC drop in your layers.
> > 
> 
> Thanks for the information.  Just a few more questions to make sure I get this right.  We have a local Git server which has clones of all the Yocto/oe layers that we are using.  So your suggestion for the example scenario would be to:
> 1. Remove PRINC reference from our .bbappend
> 2. Change .inc in mainline recipe to INC_PR = "r12"
> 3. Push changes to our local Git repositories.
> 
> Then if later I were to merge in updates from the official layers, there would be a merge conflict and I would know at merge time that I need to inspect INC_PR for being the correct value.  Does that sound right?

Yes that's possible, a bit of pain to maintain it, but as you say, at
least you get the warning in form of merge conflict. Just keep in mind
that you can drop these PR, INC_PR bumps when the upstream recipe
increments PE or PV so you can at least drop some of them in time.

> > For PR service migration the more important part is to correctly migrate
> > LOCALCOUNT numbers (used in SRCPVs), especially if you're also changing
> > package architecture (e.g. t2 suffix added in daisy or selecting different
> > DEFAULTTUNE and all your LOCALCOUNTs in PR server DB will reset to 0,
> > because of different db key to find them. Luckily you can pre-populate them
> > with single SQL command if you realize this situation before first build.
> > 
> 
> OK, now I'm confused.  I see no reference to LOCALCOUNT anywhere in the Yocto Project Mega Manual.  The only reference I see to LOCALCOUNT is on the PR Service wiki which states that you can call "bitbake-prserv-tool migrate_localcount" to migrate LOCALCOUNTs to AUTOINCs (and AUTOINC is another variable I cannot find mentioned anywhere else).  So I don't fully understand how to recognize if I need to migrate LOCALCOUNT numbers or what the SQL command would be to perform the migration.  But it sounds like this is a pretty important piece to get right during migration.  Could you explain this piece for PR service migration in more detail, or point me to the proper place in the documentation that describes this part of migration?

The important question is if you're using SRCPV in some recipes you care
about upgrade-path (e.g. linux-yocto is using it).

LOCALCOUNT was name of variable which had the N number in SRCPV
(e.g. "gitrN+<hash>"), originally it was stored in separate sqlite
database cache/bb_persist_data.sqlite3 and maintenance of this number
was moved also to PR service, that's why it's not referenced in current
documentation anymore.

AUTOINC has the same meaning, basically the fetcher will ask PR service:
"do you have AUTOINC number for revision "deadcafe1234", recipe "abc",
architecture "cortexa9t2-neon" and PR service will reply either with
already stored number or max+1 for given recipe+architecture tuple and
store the returned number.

It's important for the migration, because if you depend on N numbers in
gitrN to be from always increasing sequence you'll get zero for all
recipes if you don't migrate your old cache/bb_persist_data.sqlite3 to
PR service's sqlite db.

Honestly I did that all and it was sort of working after few manual
fixes, but I've already gave up on trusting and testing upgrade paths :/.

It's possible to use public PR service with multiple builders, but there
are some cases where the current design wont work and other cases where
it just isn't reliable enough to trust it - if you want reliable images
on target you have to reflash them from time to time (or really know
what you're doing and what you're expecting to get every time you type
"opkg update && opkg upgrade" and most end-users don't know).

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-devel/attachments/20141108/9133be87/attachment-0002.sig>