[OE-core] [PATCH] linux-yocto_4.14.bb: fix for deterministic srcversion

[Bruce Ashfield]

On Fri, Mar 30, 2018 at 7:16 PM, Khem Raj <raj.khem at gmail.com> wrote:
> On Fri, Mar 30, 2018 at 1:51 PM, Juro Bystricky
> <juro.bystricky at intel.com> wrote:
>> "srcversion" field inserted into module modinfo section contains a
>> sum of the source files which made it. However, this field can
>> be incorrect. Building the same module can end up having inconsistent
>> srcversion field eventhough the sources remain the same.
>>
>> This basically negates the whole purpose of the field srcversion,
>> and breaks build reproducibility as well.
>>
>> The problem is fairly easy reproduceable by comparing "srcversion" of
>> kernel modules built in a workplace of a short directory name with
>> "srcversion" of the same modules built in a workplace of a fairly long
>> directory name.
>>
>> The reason for incorrect srcversion is that some source files can be
>> simply silently skipped from the checksum calculation due to limited
>> buffer space for line parsing.
>>
>> [YOCTO #12544]
>>
>> Signed-off-by: Juro Bystricky <juro.bystricky at intel.com>
>> ---
>>  .../modpost-srcversion-sometimes-incorrect.patch   | 48 ++++++++++++++++++++++
>>  meta/recipes-kernel/linux/linux-yocto_4.14.bb      |  4 +-
>>  2 files changed, 51 insertions(+), 1 deletion(-)
>>  create mode 100644 meta/recipes-kernel/linux/files/modpost-srcversion-sometimes-incorrect.patch
>>
>> diff --git a/meta/recipes-kernel/linux/files/modpost-srcversion-sometimes-incorrect.patch b/meta/recipes-kernel/linux/files/modpost-srcversion-sometimes-incorrect.patch
>> new file mode 100644
>> index 0000000..1680293
>> --- /dev/null
>> +++ b/meta/recipes-kernel/linux/files/modpost-srcversion-sometimes-incorrect.patch
>> @@ -0,0 +1,48 @@
>> +"srcversion" field inserted into module modinfo section contains a
>> +sum of the source files which made it. However, this field can
>> +be incorrect. Building the same module can end up having inconsistent
>> +srcversion field eventhough the sources remain the same.
>> +This can be reproduced by building modules in a deeply nested directory,
>> +but other factors contribute as well.
>> +
>> +The reason for incorrect srcversion is that some source files can be
>> +simply silently skipped from the checksum calculation due to limited
>> +buffer space for line parsing.
>> +
>> +This patch addresses two issues:
>> +
>> +1. Allocates a larger line buffer (32k vs 4k).
>> +2. Issues a warning if a line length exceeds the line buffer.
>> +
>> +Upstream-Status: Submitted [https://patchwork.kernel.org/patch/10318141/]
>> +Signed-off-by: Juro Bystricky <juro.bystricky at intel.com>
>> +
>> +diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
>> +index 9917f92..955f26e 100644
>> +--- a/scripts/mod/modpost.c
>> ++++ b/scripts/mod/modpost.c
>> +@@ -385,9 +385,10 @@ void *grab_file(const char *filename, unsigned long *size)
>> +   * spaces in the beginning of the line is trimmed away.
>> +   * Return a pointer to a static buffer.
>> +   **/
>> ++#define MODPOST_MAX_LINE 32768
>> + char *get_next_line(unsigned long *pos, void *file, unsigned long size)
>> + {
>> +-      static char line[4096];
>> ++      static char line[MODPOST_MAX_LINE];
>> +       int skip = 1;
>> +       size_t len = 0;
>> +       signed char *p = (signed char *)file + *pos;
>> +@@ -402,8 +403,11 @@ char *get_next_line(unsigned long *pos, void *file, unsigned long size)
>> +               if (*p != '\n' && (*pos < size)) {
>> +                       len++;
>> +                       *s++ = *p++;
>> +-                      if (len > 4095)
>> ++                      if (len > (sizeof(line)-1)) {
>> ++                              warn(" %s: line exceeds buffer size %zu bytes\n"
>> ++                                   , __func__, sizeof(line));
>> +                               break; /* Too long, stop */
>> ++                      }
>
> this should be upstreamed into kernel as well.

Yup. Juro had this in the actual patch:

Upstream-Status: Submitted [https://patchwork.kernel.org/patch/10318141/]

So we can watch its progress as well.

Bruce

> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"