[OE-core] Automatic prebuilt management

[Andre McCurdy]

On Mon, Aug 5, 2019 at 9:06 AM Loic Poulain <loic.poulain at linaro.org> wrote:
> Say a company works with an OE internal tree, with open-source and closed-source packages. The company wants to release the tree so that its customers can fully customize the images/distro. The closed sources being only available internally, the company has to generate and handle prebuilt binaries for proprietary packages. Ideally, with a unified public/internal OE tree (same recipes for internal/public build).
>
> This question is actually similar to an older thread:
> https://marc.info/?l=openembedded-core&m=146779329804683

That very old thread was also referenced recently in a follow up where
I shared a later solution, see:

  http://lists.openembedded.org/pipermail/openembedded-core/2019-July/284896.html

> I started to work on this and added a 'generate-prebuilt' class which generates a tarball of ${D} in deploy/prebuilts after do_install task. Symmetrically, It's also possible to create an 'install-prebuilt' class which bypasses do_fetch, do_unpack, ..., do_install with noexec flag and instead uncompresses a previously generated prebuilt tarball into ${D} and continue normally. But I would like something smarter, e.g. first trying to check if the SRC_URI is available, if not switching on using the prebuilt package if available (e.g. in a DIR_PREBUILT)...
>
> Before going further is there already an existing solution for that? do you have any recommendation on the easier/best way to achieve this?

I did initially try the approach of having a single recipe which can
automatically support both building from source and extracting a
prebuilts tar file, but that (for me anyway) turned out to be a dead
end. Building from source requires build dependencies and config
options but extracting a prebuilt tar file does not, so the two end up
sharing very little... so 90% of the recipe ends up being conditional
on which mode it's running in. The solution I ended up with (see link
above) was for the class which creates the prebuilt tar file to also
create a dedicated mini recipe to extract it.

>From my experience however an equally hard part of the problem is the
distribution of prebuilts. It starts off easy (you share a tar file
via email or an ftp site and the receiver manually copies to their
downloads directory...) but that doesn't scale if you need to make
regular updates. My solution is discussed a little more in the thread
linked to above.