[OE-core] [PATCH] bitbake: cooker: Ensure bbappends are found in stable order

[Mark Hatle]

On 9/11/19 12:07 PM, Peter Kjellerstedt wrote:
>> -----Original Message-----
>> From: openembedded-core-bounces at lists.openembedded.org <openembedded-
>> core-bounces at lists.openembedded.org> On Behalf Of Wes Lindauer
>> Sent: den 28 augusti 2019 23:38
>> To: openembedded-core at lists.openembedded.org
>> Subject: [OE-core] [PATCH] bitbake: cooker: Ensure bbappends are found
>> in stable order
>>
>> Thanks to wildcards in bbappend filenames, it's possible to have
>> multiple bbappends that apply to the same recipe in the same directory.
>> In order to get sstate hits between different workspaces, we want to
>> apply those bbappend files in a consistent order.  Since readdir()
>> returns files in a non-deterministic order between workspaces (based on
>> inode number and/or time of creation), we'll need to sort its result in
>> order to have any consistency.
>>
>> Signed-off-by: Wes Lindauer <wesley.lindauer at gmail.com>
>> ---
>>  bitbake/lib/bb/cooker.py | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
>> index 0607fcc708..a9ee142aa5 100644
>> --- a/bitbake/lib/bb/cooker.py
>> +++ b/bitbake/lib/bb/cooker.py
>> @@ -1869,6 +1869,7 @@ class CookerCollectFiles(object):
>>              (bbappend, filename) = b
>>              if (bbappend == f) or ('%' in bbappend and bbappend.startswith(f[:bbappend.index('%')])):
>>                  filelist.append(filename)
>> +        filelist.sort()
>>          return filelist
>>
>>      def collection_priorities(self, pkgfns, d):
>> --
>> 2.14.5
> 
> I don't think this was a good idea at all. Before, bbappends were applied 
> in the priority order of the layers, but this change throws the sorting 
> based on priority out the window and might cause bbappend files for higher 
> layers to be applied before lower layers' bbappend files.
> 
> Please revert this change, and rethink how to maintain a stable order for 
> bbappend files from the same directory without affecting bbappend files 
> from different layers.

I had assumed that the sorting was only within each layer's bbappends.  If it's
global, you are absolutely right that priority order matters.

So maybe:

Within a layer (multiple .bbappends) 'sort' (alphabetic)

Between layers first by priority, then by order in the bblayers (for two layers
with the same priority)?

--Mark

> //Peter
>