Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Thu, Jul 05, 2018 at 10:31:10AM +0200, Juergen Gross wrote:
> On 05/07/18 09:53, Wei Liu wrote:
> > On Wed, Jul 04, 2018 at 03:26:16PM +0000, George Dunlap wrote:
> >>
> >>
> >>> On Jul 3, 2018, at 11:07 AM, Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> >>>
> >>> On Mon, Jul 02, 2018 at 06:03:39PM +0000, Lars Kurth wrote:
> >>>> We then had a discussion around why the positive benefits didn't 
> >>>> materialize:
> >>>> * Andrew and a few other believe that the model isn't broken, but that 
> >>>> the issue is with how we 
> >>>>   develop. In other words, moving to a 9 months model will *not* fix the 
> >>>> underlying issues, but 
> >>>>   merely provide an incentive not to fix them.
> >>>> * Issues highlighted were:
> >>>>   * 2-3 months stabilizing period is too long
> >>>
> >>> I think one of the goals with the 6 month release cycle was to shrink
> >>> the stabilizing period, but it didn't turn that way, and the
> >>> stabilizing period is quite similar with a 6 or a 9 month release
> >>> cycle.
> >>
> >> Right, and I think this was something that wasn’t quite captured in Lars’ 
> >> summary.
> >>
> >> Everyone agreed:
> >> 1. The expectation was that a shorter release cycle would lead to shorter 
> >> stabilization periods
> >> 2. This has not turned out to be the case, which means
> >> 3 At the moment, our “time doing development” to “time fixing bugs for a 
> >> release” ratio is far too low.
> >>
> >> One option to fix #3 is to go back to a 9-month cycle (or even a
> >> 12-month cycle), which would increase the “development” part of the
> >> equation.
> > 
> > You get more changes in, you also get more bugs.  Assuming bugs are
> > introduced at a constant rate in relation to changes, moving back to 9
> > months won't help.
> 
> Uuh, why not? It isn't as if no bugs are found and corrected in the
> development period. As long as the development period is longer than
> the average time between OSSTEST pushs the stabilization period should
> be roghly constant. So a longer development period will result in a
> better ratio development / stabilization.
> 
> > At least in my experience, a majority of time during the freeze is spent
> > on *waiting*. Waiting for osstest to turn around, waiting for security
> > issues to become public. Moving to 9 months won't change those factors.
> 
> But waiting isn't a factor, it is a constant, assuming the number of
> unresolved bugs at the end of the development period is roughly the
> same. And that will be the case if we don't:
> 
> a) pay no attention to OSSTEST results during development (i.e. trying
>    to get pushs as often as possible)
> 
> b) rush most series in at the end of the development period

b) has never hold in the last 5 years.

What stops committers from rushing most series at the end of the
development period? If RM is to take over the tree before stabilisation
begin, shouldn't we just call the point of takeover the start of
stabilisation period?

(IIRC the record I know is committers pushed 250 patches in a single day
before the freeze)

> 
> > A typical bug would need five working days (one week) to fix.
> > 
> > 1. Someone report or osstest reports a bug. (Day 1)
> > 2. Someone analyses it and writes a patch. (Day 2)
> > 3. Someone reviews it. (Day 2 or 3).
> > 4. Someone commits it. (Day 3 or 4).
> > 5. Osstest produces test results (Day 3 to 5).
> > 
> > For a simple bug, we might finish 1-4 in one day. But we still need to
> > allow for at least two days to get a push.
> 
> In case of OSSTEST often enough multiple bugs are reported in parallel
> and can (and should) be processed concurrently.

It is more the case that one incomplete fix blocks all other valid
fixes, so the time from staging to msater is even longer.

(The record in this case is 100 patches between staging and master and
exactly 1 calendar month to get a push)

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel