[oe] Staging - time to end the current mess?

[Koen Kooi]

On 02-11-09 16:00, Richard Purdie wrote:
> On Mon, 2009-11-02 at 15:29 +0100, Koen Kooi wrote:
>> On 02-11-09 14:43, Richard Purdie wrote:
>>> At OEDEM two years ago I proposed radically altering the way staging
>>> worked. For those that don't remember the bad old days, the staging
>>> directory layout didn't match that of the target file system meaning
>>> every recipe needed a custom do_stage and things were a mess.
>>>
>>> We changed the layout allowing the use of the gcc/binutils sysroot
>>> options and also started widespread use of autotools_stage_all and I'd
>>> say things look much improved compared to how they were.
>>>
>>> Anyone who has looked at the packaged-staging code will probably agree
>>> there is still a way to go though - its horrendously complicated. We
>>> also do a lot of duplication. I'd like to propose a new simple way of
>>> doing things. We'd have the following work flow:
>>>
>>>
>>> [...]
>>> do_compile - up to here all as usual
>>> do_install - Everything installs into a DESTDIR (including -native)
>>> do_package - Takes a copy of the DESTDIR, applies any .la/.pc mangling
>>>                then splits into packages as usual
>>> do_populate_staging - Takes a copy of the DESTDIR, mangles, installs
>>>                into staging and creates staging packages from this
>>>
>>> Firstly, does any disagree with this approach or can we all agree its a
>>> nice objective?
>>
>> Can we make the mangling a seperate tasks? I tend to do the mangling in
>> do_compile_append to be able to do builds *on* the target as well.
>
> I just published my WIP tree:
>
> http://cgit.openembedded.net/cgit.cgi/openembedded/log/?h=rpurdie/work-in-progress
>
> and specifically:
>
> http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?h=rpurdie/work-in-progress&id=1d69a38be629a70faf65dd9f2e9db12164071bfd
>
> which means we'd start declaring "mangling" functions in their own
> right. Does that work for you?

That was exactly what I was thinking about :)

> Where we run these is then up to the core metadata so we can attach them
> to stage/install/compile as makes sense. I want to declare some
> conventions of the directories these operate on so we can make the
> functions "relocatable" by changing the input variables.
>
>> And
>>
>> * 'make install' doesn't actually work properly for lots of packages
>> (you know, the ones with handcrafted makefiles)
>>
>> But we can solve that by putting the custom do_stage() methods for those
>> in do_install(), right?
>
> Those packages will already have a custom do_install in some directory
> so we can just use them as is.

My biggest fear is these kind of recipes:

do_install() {
	oe_libinstall foo ${D}
}

do_stage() }
	oe_libinstall foo ${D}
	cp foo.h ${STAGING_INC_DIR}
}

I guess we could grep for do_stage_{ap,pre}pend to find the worst 
offenders and gradually fix up remaining offenders.

regards,

Koen