[OE-core] Deployment for machine X will remove its results from machine Y's deploy dir

[Richard Purdie]

On Thu, 2014-11-27 at 15:22 +0100, Mike Looijmans wrote:
> On 11/27/2014 02:17 PM, Richard Purdie wrote:
> > On Thu, 2014-11-27 at 05:02 -0700, Gary Thomas wrote:
> >> On 2014-11-27 01:35, Mike Looijmans wrote:
> >>> Here's an example recipe to demonstrate the issue. Save it as "deployme.bb" into a recipe dir. Then build it for two machines. Building it for one machine will remove it from the
> >>> deployment directory of the other. This problem has been bugging me for months, I had files just "disappear" mysteriously from the deploy directory and seemingly random times, and
> >>> now I finally figured out what causes it.
> >>>
> >>> (cut here)
> >>>
> >>> SUMMARY = "Demonstrate a bug in OE deployment"
> >>> DESCRIPTION = "Build this package for a machine X, then look at the image's \
> >>>    deploy directory. You'll see a deployme.txt there. Now build it for another \
> >>>    machine, e.g. "Y". The deployme.txt for machine X will have disappeared \
> >>>    from the image dir. This appears to be a bug in OE's deployment."
> >>> LICENSE = "GPLv2"
> >>> LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=4d92cd373abda3937c2bc47fbc49d690"
> >>>
> >>> inherit allarch deploy
> >>>
> >>> do_compile () {
> >>>       echo "Hello world!" > deployme.txt
> >>> }
> >>>
> >>> do_deploy () {
> >>>       install -d ${DEPLOYDIR}
> >>>       install -m 644 ${B}/deployme.txt ${DEPLOYDIR}/
> >>> }
> >>>
> >>> addtask deploy before do_build after do_compile
> >>>
> >>> (cut here)
> >>
> >> Very interesting & verified with the latest master.
> >>
> >> Have you filed a bug?  https://bugzilla.yoctoproject.org/
> >
> > Well, I'm not convinced this is a bug as such. You've created an
> > "allarch" deploy task, how would you expect this to behave?
> >
> > "allarch" means that the output from this task is universal and can be
> > used on all targets. It will therefore get run once.
> >
> > A "deploy" task is machine specific.
> >
> > What ends up happening is therefore the task has a stamp is
> > "universally" created. When you change machine, the checksum of the task
> > changes, the previous version is removed, the new version is installed.
> >
> > So in many ways the system is doing exactly what I would expect it to do
> > and it isn't a bug in that sense.
> 
> It's not a bug in the sense that it doesn't do as it was programmed to do.

Its doing *exactly* what the was designed to do. That doesn't match what
you want/expect though.

>  I understand what's happening here.
> 
> I just think that the logic here is wrong. If "deploy" is machine specific, 
> then the implicit "undeploy" should be machine specific too, right?

Well, its more complex than that.

deploy.bbclass defaults to DEPLOY_DIR_IMAGE.

DEPLOY_DIR_IMAGE defaults to ${DEPLOY_DIR}/images/${MACHINE} 

I actually put off merging the latter since I knew it would cause
issues, I just couldn't articulate all of them at the time :(.

> > The real question is how should an "allarch" + "deploy" task behave when
> > you've specified machine specific paths? Perhaps erroring would be
> > better?
> 
> That would mean that roughly all deploy tasks will fail.

I'm not sure we have many deploy+allarch tasks so I think "roughly none"
would be a better description.

deploy is usually used for bootloaders and kernels, both of which are
not allarch.

> At best they're tied 
> to MACHINE_ARCH, but never to MACHINE itself.

No, they're tried to MACHINE itself, see above.

> Would be strange to put PACKAGE_ARCH="${MACHINE}" in a recipe that clearly has 
> no dependency on machine specific things. And I wrote "${MACHINE}" here on 
> purpose.

Dropping the "allarch" would be better than that.

> I was thinking along the lines of "DEPLOY_DIR_IMAGE must have the same prefix" 
> or so.
> 
> If I knew the solution, I'd have posted a patch instead of a question or report.

Well, allarch.bbclass could override DEPLOY_DIR_IMAGE to remove
the /${MACHINE} suffix. That would appear to fix the issues you're
seeing, at the risk of having a different group of people upset that you
don't get a complete directory per machine.

It comes down to which behaviour we want. Changing MACHINE in the
definition of DEPLOY_DIR_IMAGE to PACKAGE_ARCH might be the better
solution, then it will deploy based on how specific or not specific the
resulting code it. That will likely upset certain people even more
though. The other option is to accept that its machine specific and do
PACKAGE_ARCH="${MACHINE_ARCH}" in the class. That is suboptimal for the
reasons you describe but would get the behaviour some people want.

I suspect there isn't a right answer here :(

An opinion from Martin/Koen might be useful at this point since it could
affect them more than others (as well as things like the yocto project
autobuilder output and output processing/testing/releasing).

Cheers,

Richard