Re: [Xen-devel] automation: Creating a patchwork instance to improve pre-commit build testing

[Doug Goldstein <cardoe@xxxxxxxxxx>]

On Tue, Jul 24, 2018 at 04:04:05AM -0600, Jan Beulich wrote:
> >>> On 24.07.18 at 11:43, <wei.liu2@xxxxxxxxxx> wrote:
> > On Tue, Jul 24, 2018 at 03:34:51AM -0600, Jan Beulich wrote:
> >> >>> On 24.07.18 at 11:24, <wei.liu2@xxxxxxxxxx> wrote:
> >> > On Tue, Jul 24, 2018 at 03:06:08AM -0600, Jan Beulich wrote:
> >> >> >>> On 23.07.18 at 18:40, <lars.kurth@xxxxxxxxxx> wrote:
> >> >> > # How does this impact me?
> >> >> > The contribution workflow is *not* impacted by this change, but once 
> >> >> > up and 
> >> > 
> >> >> > running the following will happen once you post a patch or patch 
> >> >> > series to 
> >> >> > xen-devel:
> >> >> > * Patchwork will take patch series from the mailing list and applies 
> >> >> > it
> >> >> > * CI/DC testing is triggered
> >> >> > * A test report will be sent as a mail to the patch or the series 
> >> >> > (aka the 00 patch of the series)
> >> >> > 
> >> >> > This does mean though that series which do not build or show other 
> >> >> > issues, 
> >> >> > will likely not be reviewed until the tests pass. This would lessen 
> >> >> > the 
> >> >> > burden on reviewers, as they will know whether the code submitted 
> >> >> > builds on a 
> >> >> > wide array of environments. 
> >> >> 
> >> >> So how are dependencies between series intended to be dealt with? It
> >> >> is not uncommon for someone to say "applies only on top of xyz". The
> >> >> implication of "will likely not be reviewed until the tests pass" seems
> >> >> unsuitable to me in such a case.
> >> >> 
> >> > 
> >> > We have been asking everyone to rebase to staging before posting a new
> >> > version for a long time.  It is natural for the bot to assume that
> >> > everything should apply on top of staging. That would provide most value
> >> > to the community.
> >> > 
> >> > For special cases like you just mention, we should aim to provide
> >> > mechanisms to manually appoint a branch to be tested.
> >> 
> >> I'm afraid I disagree again: Tools used should not be dictated. I'm
> >> using quilt, not git for my work, and hence I don't maintain any
> >> branches anywhere.
> > 
> > Alright.
> > 
> > First, I don't think I said that only git would be supported.
> > Git is the most prevalent VCS nowadays, and most developers use it, so
> > it would make sense to support it first.  If you want quilt, we can
> > certainly look into that. But I'm afraid if you don't say what you
> > specifically need, nothing can be done in that regard.
> 
> Well, if you thought of other than git, then I'm afraid I lack
> understanding of where such a "branch" should be coming from.
> My first and foremost requirement is that, as stated pretty close
> to the top, the contribution workflow be *not* impacted. Any
> setting up of anything that I'd need to do would be contrary to
> that.
> 
> > Second, it is up to individual whether they want to use a certain tool
> > or not. If you don't want to use this infrastructure for whatever
> > reason, that's OK. You're only missing out all the work in the community
> > has done, but you should be able to use your own workflow just fine.
> 
> Then I maybe misunderstood Lars'es mail: I've gained the
> impression that the picking up of patches would be automatic,
> i.e. without me telling to system to do so. As it would presumably
> send its (failure) mails back to the author, I'd expect to get what
> effectively is spam in the described case.
> 
> I'm afraid my personal bar for any such automation is pretty
> high: There must not ever be any negative effect from such an
> addition. Positive effects would of course be very welcome. I
> realize this is an unrealistic goal, but it should at least come
> close (perhaps after some initial learning phase). But this implies
> that at least in theory it is possible to come close in the first
> place, which I can't take for given with the information I've been
> provided so far.
> 
> Jan

I hope you're not advocating for no progress until the perfect system is
achieved without giving anyone the opportunity to develop a system since
its impossible to develop a perfect system in the first go.

The ultimate goal here is to take patches that are posted to the mailing
list, apply them on top of staging and build them against a variety of
compiler combos coming from different distros. The results would then be
emailed as a reply to the cover letter. The idea is that this would help
maintainers/reviewers out as they could tell the submitter that it won't
get reviewed until it at least compiles.

The first improvement to the entire system I'd like to make is automatic
code-style checking. But that is depending on clang-format landing the
Xen code-style plugin.

Would you at least agree that this would be useful to some maintainers
and something some subset of folks would like to see? This isn't
necessarily targeted at code that you personally submit but folks that
are less frequent contributors. I've seen on numerous occasions a new
contributor making a patch against an outdated branch and this would
help there for example.

--
Doug

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel