Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Doug Goldstein <cardoe@xxxxxxxxxx>]

On Thu, Jul 05, 2018 at 12:16:09PM +0100, Ian Jackson wrote:
> Juergen Gross writes ("Re: [Xen-devel] [Notes for xen summit 2018 design 
> session] Process changes: is the 6 monthly release Cadence too short, 
> Security Process, ..."):
> > We didn't look at the sporadic failing tests thoroughly enough. The
> > hypercall buffer failure has been there for ages, a newer kernel just
> > made it more probable. This would have saved us some weeks.
> 
> In general, as a community, we are very bad at this kind of thing.
> 
> In my experience, the development community is not really interested
> in fixing bugs which aren't directly in their way.
> 
> You can observe this easily in the way that regression in Linux,
> spotted by osstest, are handled.  Linux 4.9 has been broken for 43
> days.  Linux mainline is broken too.
> 
> We do not have a team of people reading these test reports, and
> chasing developers to fix them.  I certainly do not have time to do
> this triage.  On trees where osstest failures do not block
> development, things go unfixed for weeks, sometimes months.

Honestly this is where we need some kind of metrics with output that my
5-year old could decipher. The OSSTEST emails are large and overwhelming
and require a bit of time commitment to digest the volume and amount of
data.

Jenkins uses weather icons to attempt to convey if this test is
trending worse or better or successful or broken. If it fails but not
every time and the amount of failures is increasing over time then its
got storm clouds. If the amount of failures is decreasing there's a
little bit of sun peaking out.

Just some kind of dashboard which would tell me what would provide the
most value to drill into would likely go a long way. But again, this is
just an assumption and could be a time waste.

--
Doug

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel