[OE-core] [PATCH 1/1] bbclass/sstate: only allowed sstate-cache objects are allowed in a build (read-only sstate-cache)

[Richard Purdie]

On Fri, 2014-09-05 at 09:05 -0500, Mark Hatle wrote:
> On 9/5/14, 4:42 AM, Richard Purdie wrote:
> > Hi Hongxu,
> >
> > On Thu, 2014-08-21 at 10:36 +0800, Hongxu Jia wrote:
> >> The requirement is the developer who demand only the "new" software
> >> they write is allowed to be compiled from source, they only want to
> >> reuse binaries from an existed sstate-cache, if the developer makes
> >> a change that triggers a rebuild, it should be an instant error.
> >>
> >> The purpose of this is for the sstate-cache to check if the item
> >> exists or not. If it doesn't the item needs to be in a whitelist
> >> or we need to fail.
> >>
> >> In the sstate-cache code, add a checking in the return path of
> >> sstate_checkhashes. If read-only sstate-cache enable, and the
> >> recipe's ${PN} not in the ${SSTATECACHE_WHITELIST}, it trigered
> >> an instant error.
> >>
> >> ...
> >> $ bitbake db
> >> ERROR: Read-only sstate-cache is enabled, the build of
> >> "db rpm-native gcc-runtime eglibc linux-libc-headers libgcc"
> >> did not come from sstate-cache. Only the recipe listed in
> >> SSTATECACHE_WHITELIST is allowed to build from source
> >>
> >> Summary: There was 1 ERROR message shown, returning a non-zero exit code.
> >
> > Sorry for the delay in getting to this, with the other patches for the
> > release, time has been difficult.
> >
> > I have been able to take a look at this and the other locked sstate
> > patches I've had pending for some time.
> >
> > Having thought quite a bit about this, I think we really want to make
> > this functionality part of the siggen class code. Where we need to add
> > hooks, we should do so with callbacks into the siggen code.
> >
> > I've just sent out a patch which my locked code in it. Could you take a
> > look and see if we can make that approach work with your code too?
> >
> > In particular, I don't really want to have multiple whitelist type
> > variables. Could we use the SIGGEN_LOCKEDSIGS variable as the definitive
> > way to control which recipes can float and which should not?
> 
> I'm all for combining this.. however it appears the patches have two slightly 
> separate goals.
> 
> The locked sstate patch tells the system to use a specific hash value and ONLY 
> that hash value.   So if a user modified a system behavior in a way that would 
> change the behavior of the component there is no warning or error.
> 
> The readonly sstate-cache on the other hand says, to reuse the sstate cache 
> items (there of course may be more then one per recipe name with a different 
> hash based on different configurations), but will warn/error when if the system 
> configuration changes in a way the hash would have changed.
> 
> So there needs to be an interface that says which items are allowed to be built 
> from source, and which items require either the locked or readonly sstate behavior.
> 
> Something like:
> 
> SIGGEN_LOCKEDSIGS = "\
> gcc-cross:do_populate_sysroot:ro \
> eglibc:do_populate_sysroot:ro \
> eglibc:do_packagedata:ro \
> gcc-cross:do_packagedata:ro \
> "
> 
> could work to extend it, but I have a concern with this.  It only affects 
> individual recipes.  The request we've had from our customers is that once they 
> have built their environment(s), and distributed the sstate-cache to their 
> developers -- only the items they distributed are allowed to be used, unless an 
> exception is granted.
> 
> So having to maintain a SIGGEN_LOCKSIGS = variable with every possible package 
> in the system seems complicated at best (for the RO case).  They really want a 
> switch that sets everything, except exceptions as read-only.
> 
> SIGGEN_LOCKEDSIGS = "\
> *:*:ro \
> bash:*:rw \
> "
> 
> but I don't know if the glob/wildcarding would add more processing overhead then 
> is acceptable.

I guess my main point right now is more about the controlling interface
of this which I believe should be the "siggen" class. What I'd like to
try and do is standardise on that as being the interface which handles
this policy, whatever it may be. If it needs hooks adding, we can do
that like the one I added from sstate.

Whether we end up with one siggen class or we have different ones
(perhaps subclassing each other) and we switch between them for
different behaviours I'm less sure about right now.

I'll give the locked sstate interface a bit more thought based on the
above, thanks. I suspect it may be a case of adding a new subclass with
some different functionality (e.g. readonly cache), then if that is what
the user requires, we select that class using BB_SIGNATURE_HANDLER?

Richard