[OE-core] [PATCH] useradd.bbclass: Make sure users/groups exist for package_write_* tasks

Fri Apr 12 18:07:41 UTC 2019

> -----Original Message-----
> From: richard.purdie at linuxfoundation.org
> <richard.purdie at linuxfoundation.org>
> Sent: den 12 april 2019 19:03
> To: Peter Kjellerstedt <peter.kjellerstedt at axis.com>
> Cc: openembedded-core at lists.openembedded.org
> Subject: Re: [OE-core] [PATCH] useradd.bbclass: Make sure users/groups
> exist for package_write_* tasks
> 
> On Fri, 2019-04-12 at 15:51 +0000, Peter Kjellerstedt wrote:
> > > -----Original Message-----
> > > From: openembedded-core-bounces at lists.openembedded.org
> > > <openembedded-
> > > core-bounces at lists.openembedded.org> On Behalf Of Peter
> > > Kjellerstedt
> > > Sent: den 4 april 2019 16:08
> > > To: Richard Purdie <richard.purdie at linuxfoundation.org>;
> > > openembedded-
> > > core at lists.openembedded.org
> > > Subject: Re: [OE-core] [PATCH] useradd.bbclass: Make sure
> > > users/groups
> > > exist for package_write_* tasks
> > >
> > > > -----Original Message-----
> > > > From: Richard Purdie <richard.purdie at linuxfoundation.org>
> > > > Sent: den 3 april 2019 14:48
> > > > To: Peter Kjellerstedt <peter.kjellerstedt at axis.com>;
> > > > openembedded-
> > > > core at lists.openembedded.org
> > > > Subject: Re: [OE-core] [PATCH] useradd.bbclass: Make sure
> > > > users/groups
> > > > exist for package_write_* tasks
> > > >
> > > > On Wed, 2019-04-03 at 14:26 +0200, Peter Kjellerstedt wrote:
> > > > > If the populate_lic task and any of the package_write_* tasks
> > > > > need to
> > > > > run, but the package task can be restored from the sstate
> > > > > cache, then
> > > > > the fetch task, which is a dependency of populate_lic, will
> > > > > wipe out
> > > > > the RSS including any users/groups that have been created. This
> > > > > results in that the package_write_* tasks are run without any
> > > > > user/group information, which causes them to fallback to either
> > > > > use
> > > > > the root user for any unknown users/groups (rpm) or to use the
> > > > > numeric
> > > > > UIDs/GIDs (deb/ipk). Neither solution will yield correct
> > > > > packages.
> > > > >
> > > > > Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt at axis.com>
> > > > > ---
> > > > >  meta/classes/useradd.bbclass | 18 ++++++++++++------
> > > > >  1 file changed, 12 insertions(+), 6 deletions(-)
> > > > >
> > > > > diff --git a/meta/classes/useradd.bbclass
> > > > > b/meta/classes/useradd.bbclass
> > > > > index 124becd082..e32315a1af 100644
> > > > > --- a/meta/classes/useradd.bbclass
> > > > > +++ b/meta/classes/useradd.bbclass
> > > > > @@ -4,7 +4,7 @@ inherit useradd_base
> > > > >  # target sysroot, and shadow -native and -sysroot provide the
> > > > > utilities
> > > > >  # and support files needed to add and modify user and group
> > > > > accounts
> > > > >  DEPENDS_append_class-target = " base-files shadow-native
> > > > > shadow-sysroot shadow base-passwd"
> > > > > -PACKAGE_WRITE_DEPS += "shadow-native"
> > > > > +PACKAGE_WRITE_DEPS += "shadow-native shadow-sysroot base-
> > > > > passwd"
> > > > >
> > > > >  # This preinstall function can be run in four different
> > > > > contexts:
> > > > >  #
> > > > > @@ -144,7 +144,10 @@ python useradd_sysroot_sstate () {
> > > > >      task = d.getVar("BB_CURRENTTASK")
> > > > >      if task == "package_setscene":
> > > > >          bb.build.exec_func("useradd_sysroot", d)
> > > > > -    elif task == "prepare_recipe_sysroot":
> > > > > +    elif (task == "prepare_recipe_sysroot" or
> > > > > +          task == "package_write_deb" or
> > > > > +          task == "package_write_ipk" or
> > > > > +          task == "package_write_rpm"):
> > > > >          # Used to update this recipe's own sysroot so the
> > > > > user/groups are available to do_install
> > > > >          scriptfile =
> > > > > d.expand("${RECIPE_SYSROOT}${bindir}/postinst-useradd-${PN}")
> > > > >          bb.build.exec_func("useradd_sysroot", d)
> > > > > @@ -161,13 +164,16 @@ python useradd_sysroot_sstate () {
> > > > >          os.chmod(scriptfile, 0o755)
> > > > >  }
> > > > >
> > > > > -do_prepare_recipe_sysroot[postfuncs] += "${SYSROOTFUNC}"
> > > > > -SYSROOTFUNC_class-target = "useradd_sysroot_sstate"
> > > > >  SYSROOTFUNC = ""
> > > > > +SYSROOTFUNC_class-target = "useradd_sysroot_sstate"
> > > > >
> > > > > -SYSROOT_PREPROCESS_FUNCS += "${SYSROOTFUNC}"
> > > > > +do_prepare_recipe_sysroot[postfuncs] += "${SYSROOTFUNC}"
> > > > > +do_package_write_deb[prefuncs] += "${SYSROOTFUNC}"
> > > > > +do_package_write_ipk[prefuncs] += "${SYSROOTFUNC}"
> > > > > +do_package_write_rpm[prefuncs] += "${SYSROOTFUNC}"
> > > > >
> > > > > -SSTATEPREINSTFUNCS_append_class-target = "
> > > > > useradd_sysroot_sstate"
> > > > > +SYSROOT_PREPROCESS_FUNCS += "${SYSROOTFUNC}"
> > > > > +SSTATEPREINSTFUNCS += "${SYSROOTFUNC}"
> > > > >
> > > > >  do_package_setscene[depends] += "${USERADDSETSCENEDEPS}"
> > > > >  do_populate_sysroot_setscene[depends] +=
> > > > > "${USERADDSETSCENEDEPS}"
> > > >
> > > > This may be a bigger problem that just package_write_*.
> > > >
> > > > This feels like a bad thing to do since there may be other tasks
> > > > which
> > > > also rely on the user information being present. This probably
> > > > only
> > > > fixes one corner case but there are likey others :(.
> > > >
> > > > Possible solutions might be:
> > > >
> > > > a) To wipe out all sstate tasks if we're rerunning do_fetch
> > >
> > > How to do that? With some pointers I hope I can whip up a patch
> > > to do this instead.
> >
> > I would very much like to have a solution for this in Warrior. If
> > the solution provided by the patch above is not acceptable, I need
> > some help where to look for how to solve it by wiping out the
> > sstate tasks if do_fetch is being rerun.
> 
> Warrior rc1 is in the process of being built so its likely too late for
> that.

Ok, nothing to do about that.

> I am very nervous about changing key pieces of the core of the
> system so close to release too which I admit is why this hasn't been a
> top priority. If people haven't run into this and there aren't tests
> for it, how big of an issue really is it?

In most cases people are probably not aware that they have run 
into it because for most recipes in OE-Core and OpenEmbedded, 
the only user used is "root", which matches what rpm will fall 
back to using if it cannot find the user for a given UID.

However, we use a lot of users/groups in our system, and after 
we were bit by rpm falling back to using root one time too many, 
we added a patch to rpm so that it fails under these conditions 
rather than silently proceeding. It has been irritating to the 
developers, but at least we no longer end up with random 
incorrectly generated RPMs in the sstate cache that are hard to 
track down.

> I'm sorry but I don't know the answer to your question without spending
> 20 minutes looking at the code and trying to figure it out. I'm
> struggling to find a spare 20 minutes at the moment :(. I know this
> isn't the answer you want but it is what it is.
> 
> I really am totally slammed with some other issues going on which are
> complicating things even more too.

No worries. We will just use the supplied patch locally for now. 
But I would appreciate if you could file the request for some 
pointers on how to proceed under "things to do once Warrior is 
out the door and I have had a week or two to breathe"? ;)

Actually, to save you from having to remember this, I have now 
added it to Bugzilla:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13279

> Cheers,
> 
> Richard

//Peter