[OE-core] [PATCH] sanity: Improve configuration upgrade capabilities (support meta-yocto -> poky transition)

Sun Feb 28 12:20:16 UTC 2016

On Sun, Feb 28, 2016 at 11:53 AM, Richard Purdie
<richard.purdie at linuxfoundation.org> wrote:
> Right now, only one configuration file can be processed (conf/bblayers.conf)
> and it can only have one version number. This is a cause of immense friction
> between OE-Core and Poky since if one needs a version change, it shouldn't
> be forced on the other.
>
> We'd like to rename the meta-yocto layer (within the meta-yocto repository)
> to meta-poky. To do this, we need to correct the bblayers.conf file and that
> means changing the sanity version. After the pain this caused the last time,
> Paul made me promise never to have them out of sync between OE-Core and Poky,
> equally, having every distro changing config update OE-Core isn't scalable
> either.
>
> This patch changes the sanity upgrade method to list a more generic format:
>
> <config file>:<current version variable name>:<required version variable name>:<upgrade function>
>
> This in theory allows us to support upgrades to any of the core
> configuration files, and allow layers to extend them as needed. Files
> with the same name can be handled in different layers by setting a unique
> version name variable in the file itself. The upgrade code is only called
> if the version variable is set.
>
> To allow us to make the poky name change and use a new configuration file
> name, one last version bump is included for poky to handle the transition.
>
> Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
>
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 49693e7..a0553ee 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass

I think this breaks parsing. Have reverted it and can build again. I got
[superandy at localhost oe-core]$ bitbake -k xfce4-games-image
ERROR: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:oecore_update_bblayers(d)
     0003:
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 151, function: oecore_update_bblayers
     0147:            with open(bblayers_fn, "w") as f:
     0148:                f.write(''.join(lines))
     0149:            return
     0150:
 *** 0151:        sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', "7")
     0152:        return
     0153:
     0154:    raise NotImplementedError(failmsg)
     0155:}
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 24, function: sanity_conf_update
     0020:        if re.search(pattern, line)), (None, None))
     0021:
     0022:def sanity_conf_update(fn, lines, version_var_name, new_version):
     0023:    index, line = sanity_conf_find_line(version_var_name, lines)
 *** 0024:    lines[index] = '%s = "%d"\n' % (version_var_name, new_version)
     0025:    with open(fn, "w") as f:
     0026:        f.write(''.join(lines))
     0027:
     0028:# Functions added to this variable MUST throw a
NotImplementedError exception unless
Exception: TypeError: %d format: a number is required, not str

ERROR: Execution of event handler 'check_sanity_eventhandler' failed
Traceback (most recent call last):
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity(sanity_data=<bb.data_smart.DataSmart object
at 0x7f932aa55890>):

    >    check_sanity_everybuild(status, sanity_data)

  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild(status=<SanityStatus object at
0x7f932aca3810>, d=<bb.data_smart.DataSmart object at
0x7f932aa55890>):

    >    sanity_check_conffiles(status, d)

  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles(status=<SanityStatus object at
0x7f932aca3810>, d=<bb.data_smart.DataSmart object at
0x7f932aa55890>):
                 try:
    >                bb.build.exec_func(func, d)
                 except NotImplementedError as e:
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f932aa55890>, dirs=None):
             if ispython:
    >            exec_func_python(func, d, runfile, cwd=adir)
             else:
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f932aa55890>,
runfile='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/temp/run.oecore_update_bblayers.4316',
cwd='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/defaultpkgname-1.0'):
         except:
    >        raise FuncFailed(func, None)
         finally:
FuncFailed: Function failed: oecore_update_bblayers

ERROR: Command execution failed: Traceback (most recent call last):
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/command.py",
line 101, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/cooker.py",
line 1538, in updateCache
    bb.event.fire(bb.event.SanityCheck(False), self.data)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 171, in fire
    fire_class_handlers(event, d)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 110, in fire_class_handlers
    execute_handler(name, handler, event, d)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 82, in execute_handler
    ret = handler(event)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 991, in check_sanity_eventhandler
    reparse = check_sanity(sanity_data)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity
    check_sanity_everybuild(status, sanity_data)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild
    sanity_check_conffiles(status, d)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles
    bb.build.exec_func(func, d)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python
    raise FuncFailed(func, None)
FuncFailed: Function failed: oecore_update_bblayers

Summary: There were 3 ERROR messages shown, returning a non-zero exit code.
[superandy at localhost oe-core]$ bitbake -k xfce4-games-image
ERROR: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:oecore_update_bblayers(d)
     0003:
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 151, function: oecore_update_bblayers
     0147:            with open(bblayers_fn, "w") as f:
     0148:                f.write(''.join(lines))
     0149:            return
     0150:
 *** 0151:        sanity_conf_update(bblayers_fn, lines, 'LCONF_VERSION', "7")
     0152:        return
     0153:
     0154:    raise NotImplementedError(failmsg)
     0155:}
File: '/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass',
lineno: 24, function: sanity_conf_update
     0020:        if re.search(pattern, line)), (None, None))
     0021:
     0022:def sanity_conf_update(fn, lines, version_var_name, new_version):
     0023:    index, line = sanity_conf_find_line(version_var_name, lines)
 *** 0024:    lines[index] = '%s = "%d"\n' % (version_var_name, new_version)
     0025:    with open(fn, "w") as f:
     0026:        f.write(''.join(lines))
     0027:
     0028:# Functions added to this variable MUST throw a
NotImplementedError exception unless
Exception: TypeError: %d format: a number is required, not str

ERROR: Execution of event handler 'check_sanity_eventhandler' failed
Traceback (most recent call last):
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity(sanity_data=<bb.data_smart.DataSmart object
at 0x7f2bb26f3cd0>):

    >    check_sanity_everybuild(status, sanity_data)

  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild(status=<SanityStatus object at
0x7f2bb26f31d0>, d=<bb.data_smart.DataSmart object at
0x7f2bb26f3cd0>):

    >    sanity_check_conffiles(status, d)

  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles(status=<SanityStatus object at
0x7f2bb26f31d0>, d=<bb.data_smart.DataSmart object at
0x7f2bb26f3cd0>):
                 try:
    >                bb.build.exec_func(func, d)
                 except NotImplementedError as e:
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f2bb26f3cd0>, dirs=None):
             if ispython:
    >            exec_func_python(func, d, runfile, cwd=adir)
             else:
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python(func='oecore_update_bblayers',
d=<bb.data_smart.DataSmart object at 0x7f2bb26f3cd0>,
runfile='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/temp/run.oecore_update_bblayers.4459',
cwd='/home/superandy/tmp/oe-core-glibc/work/cortexa7t2hf-neon-vfpv4-angstrom-linux-gnueabi/defaultpkgname/1.0-r0/defaultpkgname-1.0'):
         except:
    >        raise FuncFailed(func, None)
         finally:
FuncFailed: Function failed: oecore_update_bblayers

ERROR: Command execution failed: Traceback (most recent call last):
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/command.py",
line 101, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/cooker.py",
line 1538, in updateCache
    bb.event.fire(bb.event.SanityCheck(False), self.data)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 171, in fire
    fire_class_handlers(event, d)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 110, in fire_class_handlers
    execute_handler(name, handler, event, d)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/event.py",
line 82, in execute_handler
    ret = handler(event)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 991, in check_sanity_eventhandler
    reparse = check_sanity(sanity_data)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 954, in check_sanity
    check_sanity_everybuild(status, sanity_data)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 768, in check_sanity_everybuild
    sanity_check_conffiles(status, d)
  File "/home/superandy/data/oe-core/sources/openembedded-core/meta/classes/sanity.bbclass",
line 548, in sanity_check_conffiles
    bb.build.exec_func(func, d)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 227, in exec_func
    exec_func_python(func, d, runfile, cwd=adir)
  File "/home/superandy/data/oe-core/sources/bitbake/lib/bb/build.py",
line 263, in exec_func_python
    raise FuncFailed(func, None)
FuncFailed: Function failed: oecore_update_bblayers