[bitbake-devel] [PATCH] data/siggen: Switch md5 -> sha256

[Mark Hatle]

On 7/23/19 11:09 AM, Mike Crowe wrote:
> On Monday 17 December 2018 at 17:16:45 +0000, Richard Purdie wrote:
>> Similarly to the codeparser change, change to sha256 hashes due to worries
>> over collisions. The main impact of this change is slightly slower parsing
>> time as well as longer sstate file names.
>>
>> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> 
> This change has given us several sstate cache filenames that exceed the 255
> character limit of ext4. :(

I wonder if another approach might be easier.  Instead use change the filename
to be:

sstate:<sstate_version>:<hash>:recipe:version:...:task.<type>

(<type> such as .tgz.signifo)

If the filename becomes too long, simply truncate the middle section) preserving:

sstate:<state_version>:<hash> and <type>

So the same truncation when matching, and it should be safe.  The recipe,
version ... task bit is nice to have for a human reader, but I don't think it's
actually needed by the system itself.

This should avoid any collisions and keep the maximum information that will fit
in the filename(s).

--Mark

> We have a recipe that constructs a source tree from three separate Git
> repositories, so we end up with a PV that is 93 characters long, which
> doesn't help.
> 
> I've found that using urlsafe_b64encode to shorten the SHA-256 hashes
> reduces the filename lengths by 21 characters, which is more than the 14
> characters we need to save. However, there's a risk of collisions if the
> cache is stored on a case-insensitive filesystem.
> 
> Would you be amenable to a patch to use urlsafe_b64encode?
> 
> If not, would you be amenable to a patch that consolidated the algorithm
> into a single function called from the places you changed so that we can
> patch it locally more easily? Or perhaps there's another way to make it
> configurable?
> 
> Thanks.
> 
> Mike.
>