[bitbake-devel] [PATCH] data/siggen: Switch md5 -> sha256

[Mark Hatle]

On 7/24/19 4:11 AM, Mike Crowe wrote:
> On Wednesday 24 July 2019 at 09:11:09 +0100, Richard Purdie wrote:
>> On Tue, 2019-07-23 at 11:59 -0500, Mark Hatle wrote:
>>> On 7/23/19 11:09 AM, Mike Crowe wrote:
>>>> On Monday 17 December 2018 at 17:16:45 +0000, Richard Purdie wrote:
>>>>> Similarly to the codeparser change, change to sha256 hashes due
>>>>> to worries
>>>>> over collisions. The main impact of this change is slightly
>>>>> slower parsing
>>>>> time as well as longer sstate file names.
>>>>>
>>>>> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org
>>>>>>
>>>>
>>>> This change has given us several sstate cache filenames that exceed
>>>> the 255
>>>> character limit of ext4. :(
>>>
>>> I wonder if another approach might be easier.  Instead use change the
>>> filename
>>> to be:
>>>
>>> sstate:<sstate_version>:<hash>:recipe:version:...:task.<type>
>>>
>>> (<type> such as .tgz.signifo)
>>>
>>> If the filename becomes too long, simply truncate the middle section)
>>> preserving:
>>>
>>> sstate:<state_version>:<hash> and <type>
> 
> We make use of the recipe name in our sstate cache pruning script, since
> some recipes have more churn and we expire them more quickly. That
> shouldn't be a problem as long as the recipe name is kept in preference to
> the version.

Ya, this is the human readable part.  However, you can still dump the contents
of the siginfo using the bitbake-dumpsig and get anything that was truncated
off.  So I would still consider it 'human readable', that happens to be machine
readable in most cases.

>>> So the same truncation when matching, and it should be safe.  The
>>> recipe,
>>> version ... task bit is nice to have for a human reader, but I don't
>>> think it's
>>> actually needed by the system itself.
>>>
>>> This should avoid any collisions and keep the maximum information
>>> that will fit
>>> in the filename(s).
>>
>> In some ways regardless of what we do, we need to abstract it so that
>> we have one function generating the filenames.
>>
>> So yes, think we need a patch which abstracts the filename generation.
>> It needs to be able to truncate elements of the constructed name other
>> than the hash. I do prefer truncating the other elements than changing
>> the hash.
> 
> If we're going to keep the 64-character SHA-256 hash and instead elide
> parts of the sstate filenames, then I think only sstate.bbclass needs to
> change, so this isn't a Bitbake issue and we should probably move this
> thread it openembedded-core.
> 
> Or perhaps I'm misunderstanding your suggestion.

I'm not sure where all of the filenames are handled, if it's in oe-core  sstate
class, then yes.. I think moving the discussion is the right next move.

It may be a joint change though, if filename generation needs an API or sync
with bitbake.  (I've not touched this part before, so I really don't know how
that part is generated.)

--Mark

> Thanks.
> 
> Mike.
>