[OE-core] PRServer's problem

Martin Jansa martin.jansa at gmail.com
Thu May 19 08:47:54 UTC 2016 

As the commit says, small change in package.bbclass also causes all
packages to be recreated with PR bump even when the content is most likely
the same.

Fixing bug in gcc may at least provide different binaries so it might be
useful to upgrade them on target (or at least distinguish if they were
already rebuilt with fixed gcc-cross or not).

Reverting the commit doesn't fix the issue that sstate handler cannot
compare if the change in signature produced "significantly different"
binaries. If it's reverted in oe-core I'll just revert the revert in our
builds, because we care about reproducible builds and we already gave up on
working upgrade paths, which are broken too often anyway.

Reflash of "system" partitions and storing user data/configuration
somewhere else is faster and safer since the sstate was invented (since we
stopped using OE classic). You can find some rant e-mails from me about
this e.g.
http://lists.openembedded.org/pipermail/openembedded-core/2011-November/051354.html
and the Yocto bug I gave you.

Regards,

On Thu, May 19, 2016 at 4:33 AM, Robert Yang <liezhi.yang at windriver.com>
wrote:

>
> Hi Martin,
>
> I found this patch in the bug:
>
>
> http://git.openembedded.org/openembedded-core/commit/meta/lib/oe/sstatesig.py?id=336a7897e39b9e42dcfcba9e2520ea96b0c6a8d6
>
> Too many PR bumps and rebuildings are caused by this patch. I'm
> not very sure about what this patch tries to fix, it seems that
> it is trying to fix problems when 32bit and 64bit uses the same
> sstate cache? Would you please provide a simple reproducer, please?
>
> Things will become much better after revert this patch. Be less
> stricter or more is a hard problem, if we still need the patch,
> can we leave such a choice to the end user? We can add some vars
> like DROP_NATIVE_SIG ? = "0/1". This would be good to the end user
> who uses stable release like jethro or krogoth to make their
> distributions, and PR Service really matters here. Even if they
> switch the build between 32 and 64 builds (This is unlikely to
> happen for production environment) and got problems, they still
> can fix the problem by rebuilding, this is still much better than
> current status: run a "smart/opkg/apt-get upgrade" on the target,
> *all* of the packages are downloaded and installed again after a
> CVE patch applies on native recipes like pseudo-native or rpm-native,
> but in fact, nothing is changed on the target this is really a bad
> experience.
>
> // Robert
>
>
> On 05/18/2016 06:15 PM, Martin Jansa wrote:
>
>> On Wed, May 18, 2016 at 05:31:09PM +0800, Robert Yang wrote:
>>
>>>
>>>
>>> On 05/18/2016 05:20 PM, Martin Jansa wrote:
>>>
>>>> On Wed, May 18, 2016 at 04:03:58PM +0800, Robert Yang wrote:
>>>>
>>>>> Hi Martin,
>>>>>
>>>>> On 05/18/2016 03:39 PM, Martin Jansa wrote:
>>>>>
>>>>>> See:
>>>>>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=5970
>>>>>>
>>>>>> Just using recipe checksum wont work, because the main reason for PR
>>>>>> bumps is to
>>>>>> automatically upgrade the packages when one of the dependencies
>>>>>> changes .so
>>>>>> version, which you won't detect from recipe checksum of the app which
>>>>>> is just
>>>>>> using the library.
>>>>>>
>>>>>
>>>>> For the development branch like master, yes, that would happen. But for
>>>>> the stable release like jethro and krogoth, it is unlikely that would
>>>>> happen, and if if does, the user can manually bump the impacted
>>>>> recipe's
>>>>> PR to fix the problem. The current problem is that when *all* recipes'
>>>>> PR are bumped, there is no way to fix the problem.
>>>>>
>>>>
>>>> You can still stop using PR service and start doing manual PR bumps, but
>>>>
>>>
>>> We can't stop PR service and start doing manual PR bumps since we need
>>> keep
>>> update to date with upstream, the changes from upstream don't do the
>>> manual
>>> PR, and I don't think that we have to if they can be done automatically.
>>>
>>> it's quite annoying if you need to bump a lot of recipes you don't
>>>> control :).
>>>>
>>>
>>> What's your opinion about only consider RDEPENDS for PR service's
>>> checksum,
>>> please ?
>>>
>>
>> I would like to have separate handler as described in that bug, option
>> c).
>>
>> Not only because of unnecessary EXTENDPRAUTO bumps, but also unnecessary
>> rebuilds.
>>
>> On Wed, May 18, 2016 at 8:09 AM, Robert Yang <liezhi.yang at windriver.com
>>>>>> <mailto:liezhi.yang at windriver.com>> wrote:
>>>>>>
>>>>>>       The PRServer bumps PR according to do_package's task hash, that
>>>>>>       causes it bumps *all* packages' PR when recipes like
>>>>>> pseudo-native
>>>>>>       and rpm-native is changed. It is a very bad user experience
>>>>>> when we
>>>>>>       run "smart/opkg upgrade" on running target, for example, when
>>>>>> we apply
>>>>>>       a CVE patch to pseudo-native or rpm-native, or do some slight
>>>>>> changes
>>>>>>       in their do_compile, "smart/opkg upgrade" will download/install
>>>>>> *all*
>>>>>>       the packages since all of the packages' PR are bumped.
>>>>>>
>>>>>>       Here are some rough suggestions to fix this problem, and please
>>>>>> feel
>>>>>>       free to give your suggestions.
>>>>>>       1) Do not use do_package's task for bumping PR, the easiest way
>>>>>>           is simulate manually bump PR -- only bump PR when the recipe
>>>>>>           itself's checksum is changed.
>>>>>>
>>>>>>       2) Add a new task for PRServer, redefine its task hash for
>>>>>> bumping
>>>>>>           PR, for example, this task hash only considers RDEPENDS (no
>>>>>>           DEPENDS), and drop any native dependencies.
>>>>>>
>>>>>>       I prefer the first way, and an alternative way maybe add a var
>>>>>> so that
>>>>>>       the user can configure it:
>>>>>>       PR_CHECKSUM = "${BB_TASKHASH}" (current way)
>>>>>>       Or
>>>>>>       PR_CHECKSUM = "<recipe checksum>"
>>>>>>
>>>>>>       --
>>>>>>       Thanks
>>>>>>
>>>>>>       Robert
>>>>>>       --
>>>>>>       _______________________________________________
>>>>>>       Openembedded-core mailing list
>>>>>>       Openembedded-core at lists.openembedded.org
>>>>>>       <mailto:Openembedded-core at lists.openembedded.org>
>>>>>>
>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>>
>>>>>>
>>>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20160519/61f55231/attachment-0002.html>

More information about the Openembedded-core mailing list