[oe] Still many meta-networking changes waiting in patchwork/master-next

Thu Mar 3 20:46:46 UTC 2016

[Re: [oe] Still many meta-networking changes waiting in patchwork/master-next] On 16.03.03 (Thu 11:08) Paul Eggleton wrote:

> On Wed, 02 Mar 2016 08:27:26 Joe MacDonald wrote:
> > [Re: [oe] Still many meta-networking changes waiting in patchwork/master-
> next] On 16.03.02 (Wed 11:33) Martin Jansa wrote:
> > > On Fri, Feb 26, 2016 at 08:39:45PM -0500, Joe MacDonald wrote:
> > > > Hi Martin,
> > > > 
> > > > [Re: [oe] Still many meta-networking changes waiting in 
> patchwork/master-next] On 16.02.26 (Fri 20:56) Martin Jansa wrote:
> > > > > On Fri, Feb 26, 2016 at 01:13:17PM -0500, Joe MacDonald wrote:
> > > > > > Hey Martin,
> > > > > > 
> > > > > > [[oe] Still many meta-networking changes waiting in 
> patchwork/master-next] On 16.02.25 (Thu 18:19) Martin Jansa wrote:
> > > > > > > Hi Joe.
> > > > > > 
> > > > > > > there are still 18 meta-networking commits in master-next:
> > > > > > I thought I'd follow up to this and let you know I'll do something
> > > > > > with
> > > > > > these before the end of the month.  Almost all of them (waf-samba
> > > > > > aside)
> > > > > > obviously got missed because I'm watching patchwork based on
> > > > > > 'meta-networking' and none of these had it in the subject line.
> > > > > > 
> > > > > > I think I've asked before, but are you curating all of those bundles
> > > > > > by
> > > > > > hand or have you got some machinery that filters patches into
> > > > > > bundles
> > > > > > based on path names in the patches?  Because that would be kind of
> > > > > > neat
> > > > > > to have.
> > > > > 
> > > > > I think I've answered before, but I'm filtering them manually based on
> > > > > subject and also actual paths inside the patch. I also need to mark
> > > > > them
> > > > > "Accepted", "Superseded", ..  manually, there is git hooks which is
> > > > > supposed to mark at least accepted one, but it finds 1 from 1000 if
> > > > > any.
> > > > 
> > > > Yeah, I've seen the hooks trying to find something when I push commits
> > > > and
> > > > just spewing a slew of errors.  I generally do my best to stay on top of
> > > > the status in patchwork, doing it all at one time, though.
> > > > 
> > > > > This is of course a bit error prone, especially when there are
> > > > > multiple
> > > > > versions of the same patch already in master-next - I usually end up
> > > > > marking all versions merged patch as Accepted (unless I've marked
> > > > > older
> > > > > versions already as Superseded when filtering incoming queue).
> > > > 
> > > > I took a bit of time this afternoon to try to come up with a way to
> > > > automate at least part of the process for me and I'm sure there's a
> > > > better
> > > > way to do it (hence why I asked again) but since it sounds like you're
> > > > doing it by hand for a much larger space than I am, maybe there's not.
> > > > 
> > > > Anyway, for what it's worth, here's what I came up with:
> > > >    git log --cherry-pick --format="'%s'" \
> > > >    
> > > >       oe/master-next...oe/master meta-networking | \
> > > >       xargs -r -n1 pwclient search -f "%{id} %{name}" -s New
> > > > 
> > > > Which does pretty well, though it obviously goes a little insane if
> > > > someone puts a ' in the short log, but it didn't seem worth trying to
> > > > work
> > > > around that pretty rare (I hope) corner case.  As an aside, the first
> > > > time
> > > > I did this I was using --format="\"%s\"" and the *very* top commit
> > > > ('recipes: Replace "cp ...) showed me what kind of pain I'm in for when
> > > > there are colliding characters in the short log.  It's ugly but nothing
> > > > catastrophic.
> > > > 
> > > > So that gave me 14 of the patches you asked about.  The others were easy
> > > > 
> > > > to find:
> > > >    git log --cherry-pick --format="'%s'" \
> > > >    
> > > >       oe/master-next...oe/master meta-networking | \
> > > >       xargs -r -n1 pwclient search -f "%{id} %{name}" -s Accepted
> > > > 
> > > > But obviously they haven't been accepted into 'master' next, obviously
> > > > just a typo or a mis-click at some point.  So that's not bad.
> > > > 
> > > > Once I had that sanity check done, it's easy to harvest them all:
> > > >    git log --cherry-pick --format="'%s'" \
> > > >    
> > > >       oe/master-next...oe/master meta-networking | \
> > > >       xargs -r -n1 pwclient search -f "%{name}" -s new | \
> > > >       sed 's=\[.*\] *==;s="=.=g;s=\(.*\)="\1"=' | \
> > > >       xargs -r -n1 git log --format="%h" --grep | \
> > > >       xargs -r -n1 git cherry-pick -s
> > > > 
> > > > The hideous sed in the middle is just to throw out stuff from the
> > > > pwclient
> > > > output that doesn't appear in the git logs (eg. "[v2]") and to skip over
> > > > the craziness that happens on the 'git log grep' if you have a " in the
> > > > subject.
> > > > 
> > > > Run it a second time to grab the three 'accepted' patches and we're
> > > > nearly
> > > > done.
> > > > 
> > > > Setting aside all of the inspection steps that follow that nobody would
> > > > want to automate, the same machinery applies equally well to keeping
> > > > 
> > > > patchwork up to date:
> > > >    git log --cherry-pick --format="'%s'" \
> > > >    
> > > >       master...oe/master meta-networking | \
> > > >       xargs -r -n1 pwclient search -f "%{id}" -s new | \
> > > >       xargs -r -n1 pwclient update -s "accepted"
> > > > 
> > > > There's not really any point in running this one a second time for the
> > > > patches already marked 'accepted'.
> > > > 
> > > > I've been using a version of this last one for a while now because the
> > > > git
> > > > hooks are non-functional.
> > > > 
> > > > The end result is that my semi-automated process above gets 17 of the 18
> > > > patches you cited and the 18th (waf-samba.bbclass) is a special case
> > > > that
> > > > I don't think could ever be caught except by manual intervention.
> > > > 
> > > > Mostly just throwing this out there so that maybe it'll help you or
> > > > someone else with similar tasks and maybe someone can look at what I'm
> > > > doing and point out obvious flaws / shortcomings / bear-traps /
> > > > improvements.  And also since I haven't bothered to put this into a
> > > > shell
> > > > function or git alias yet, at least my process is archived in the
> > > > mailing
> > > > list and I can find it again if I need it.
> > > 
> > > Thanks for trying to improve patchwork experience with bunch of scripts.
> > > Personally I would prefer to just use gerrit (that's why I gave up
> > > trying to work around patchwork issues with scripts and rather sort &
> > > apply the patches manually with just small help from pwclient).
> > 
> > I've never been a fan of gerrit, but I've only used it on a couple of
> > projects, so I don't really have a lot of experience with it.  Probably
> > obviously, so long as the CLI experience isn't terrible and I can easily
> > script around things that don't work well for me, I can work with almost
> > anything.
> 
> FWIW, and I know some people may be tired of hearing about this without much 
> visible progress, but we still have some people within Intel working on 
> Patchwork as well as some automated patch testing functionality aimed at OE 
> specifically, and they are making progress. I hope to have something to show 
> pretty soon. It should eliminate a lot of the manual work required with the 
> current version.

I was wondering about that.  I'm quite looking forward to it.  At the
very least I know newer versions of patchwork than the one we have right
now offer an improved bundle experience.

-- 
-Joe MacDonald.
:wq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-devel/attachments/20160303/1b00745a/attachment-0002.sig>