[bitbake-devel] runqueue: Ensure task environment is correct

[Richard Purdie]

On Wed, 2011-09-07 at 14:43 -0700, Chris Larson wrote:
> On Wed, Sep 7, 2011 at 2:11 PM, Richard Purdie
> <richard.purdie at linuxfoundation.org> wrote:
> > On Wed, 2011-09-07 at 11:38 -0700, Chris Larson wrote:
> >> On Wed, Sep 7, 2011 at 10:43 AM, Richard Purdie
> >> <richard.purdie at linuxfoundation.org> wrote:
> >> > This fixes two problems:
> >> >
> >> > a) Variables which were in the parent environment but not set as "export"
> >> >   variables in the datastore could end up in the task environment
> >> >
> >> > b) oe.environ.update() can't cope with the generator returned by
> >> >   bb.data.exported_vars()
> >> >
> >> > Whilst the updated code isn't as neat, it does do the expected thing,
> >> > sets the environment correctly and stops unwanted values leaking into
> >> > the task environment.
> >> >
> >> > Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> >> >
> >> > diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> >> > index 5a4321f..72e6845 100644
> >> > --- a/bitbake/lib/bb/runqueue.py
> >> > +++ b/bitbake/lib/bb/runqueue.py
> >> > @@ -1136,7 +1136,12 @@ class RunQueueExecute:
> >> >                 for h in self.rqdata.hash_deps:
> >> >                     the_data.setVar("BBHASHDEPS_%s" % h, self.rqdata.hash_deps[h])
> >> >
> >> > -                os.environ.update(bb.data.exported_vars(the_data))
> >> > +                # exported_vars() returns a generator which *cannot* be passed to os.environ.update()
> >> > +                # successfully. We also need to unset anything from the environment which shouldn't be there
> >> > +                exports = bb.data.exported_vars(the_data)
> >> > +                bb.utils.empty_environment()
> >> > +                for e, v in exports:
> >> > +                    os.environ[e] = v
> >>
> >> os.environ.clear()
> >> os.environ.update(bb.data.exported_vars(the_data))
> >
> > I tried this when testing and when I read os.environ back, it was not
> > correct. There is something odd occurring if you pass the generator into
> > update() directly.
> 
> Odd. It works fine in my local testing. Maybe there's a python bug
> lurking -- I can't imagine it's supposed to work this way. Guess we
> have to do it this way, but I really think you should consider
> isolating the problem and reporting it upstream to the python project.

Agreed, it does seem to be something about my machine/python version
since this is not the first time I've had issues with os.environ :(.

Which version did you test with? I was seeing this with Ubuntu's python
"2.7.1+" whatever that is. I did try and write a small test case and
that works which is odd/worrying/annoying...

Cheers,

Richard