[OE-core] Patch merge process

[Khem Raj]

> On Sep 1, 2015, at 1:04 PM, Richard Purdie <richard.purdie at linuxfoundation.org> wrote:
> 
> On Tue, 2015-09-01 at 10:40 -0700, Khem Raj wrote:
>> I thought of bringing this up for discussions, I am seeing that, in
>> some pull requests some patches get dropped and I understand
>> that there might be a reason for that but that reason is not
>> communicated in proposed patch mail often. I have experienced with
>> some of the
>> pull requests myself and seen it with some others. I think we are not
>> responding to patches as it should be. Since we follow
>> just mailing list as upstreaming process thats the only channel that
>> developer is engaged w.r.t. patches and there should be a conclusion
>> on the mailing list itself for a given patch. So I would request to
>> whoever along with Richard is validating the patches and bringing them
>> to OE-Core from mailing lists to respond to the patches especially the
>> ones that are not applied with reasons why its not applied and
>> similarly to the ones that are applied but has been rejected by some
>> reviewers. It would also be good to document the commit criteria
>> somewhere, so developer can do rework and ease the pressure on
>> maintainers. Can we explore some tools that can help here ? May be
>> failed errors.yp.org output can be sent to list of developers whose
>> patches has been part of staging ?
>> 
>> I think its extremely important to keep a developer informed and
>> engaged about the upstreaming if we want to encourage more voluntary
>> participation in the project.
>> 
>> It would be good to identify where we have bottlenecks and how can we
>> improve upon the situation, ideas are welcome.
> 
> The problem basically comes down to this. We have:
> 
> * two people who put patches together into blocks and try and test them
> * a single autobuilder instance capable of running the tests we need to
>  verify a set of patches

will results from other auto builders reporting to errors.yp.org can create some confidence

> * test runs that take around 10 hours (substantially increased recently
>  by more test automation)
> * ever increasing expectations that OE-Core master doesn't break

> * ever increasing problems of 'random' autobuilder failures which seem
>  related to auto builder load


I understand these issues, however response would engage the developers and keep them in loop even if it is delayed.

> 
> Put all this together and its turning into a nightmare to keep things
> flowing. The increased QA automation is catching a lot more issues which
> is good, but its also making the cycles longer. The increased load is
> also shaking out all kinds of races and qemu issues.
> 
> There is a SWAT team who is meant to help try and take the load off
> Ross/I for some of the process. The processes there are being changed to
> try and better help Ross/I. I'd ask anyone reading this and frustrated
> with the patch merge process, when did you last help with SWAT? When did
> you last help with any of the random autobuilder failures we're seeing
> or with general YP bug fixing (not just bugs you yourself run into)?

I think there could be better message reports that can be sent to mailing list on auto builder failures. Many devs
may not know about the YP’s build schedules and nomenclature.

I am particularly talking about OE-core, many devs may not be involved with Yocto project activities, even though those
activities may improve the quality of OE-core. Its just the reality.
many developers may folks chimes in when it breaks them with patches or reports and thats not a bad response, It
would be much worse if they abandoned to use the project.

> 
> One problem is that we have to test the patches in a block due to the
> long test cycle. If something fails, we can't know who broke it, that is
> up to someone's best guess so it can't be automated. Yes, you could send
> out block emails but those tend to confuse people. Heck, even getting
> the autobuilder to send email at all at the moment is proving
> problematic. I also literally spent two days last week trying to fix
> basic issues ensuring the autobuilder actually builds the revision we
> think its building!

I agree CI is not easy while you are also doing system integration iteratively.
while the build infra issues would remain, some of these concerns can be addressed if we were to use some code review tool
which can interface with auto builders and provide quick smoke tests on per patch bases, with sstate in place this could be achieved in minutes.

> 
> We are trying to send out info where and when things break and we know
> the cause. Equally, some things have been removed as "best guess" and we
> sometimes forget to either add them back, or follow up with a reply
> about a likely failure they introduced :(. It comes down to simply being
> overworked. Equally, the work in doing this is to be quite honest pretty
> thankless and some of the lack of basic testing sometimes seen doesn't
> really help :(.

I think its because the process is quite manual probably, you could send an auto email to patch when its picked to a staging branch
and when its deleted/changed, many review tools do this automatically. From personal experience I now believe that with a code review tool we can have lot more developers participate concretely in the code flight and since it provides good staging it scales quite well. So I would think using some review tool might help, at this time

> 
> That all said, I will do my best to try and send more informative emails
> about patches…

appreciated.

> 
> Cheers,
> 
> Richard
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 204 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20150901/a9a26986/attachment-0002.sig>