Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

On 05/07/18 10:43, Roger Pau Monné wrote:
> On Thu, Jul 05, 2018 at 09:19:10AM +0100, Wei Liu wrote:
>> On Thu, Jul 05, 2018 at 10:06:52AM +0200, Roger Pau Monné wrote:
>>> On Thu, Jul 05, 2018 at 08:53:51AM +0100, Wei Liu wrote:
>>>> On Wed, Jul 04, 2018 at 03:26:16PM +0000, George Dunlap wrote:
>>>>> So a fair amount of the discussion was about what it would look like,
>>>>> and what it would take, to make it such that almost any push from
>>>>> osstest (or whatever testing infrasctructure we went with) could
>>>>> reasonably be released, and would have a very low expectation of
>>>>> having extraneous bugs.
>>>>
>>>> I would also like to advocate changing the mentality a bit. The current
>>>> mentality is that "we want to be reasonably sure there is low
>>>> expectation of bugs before we can release". Why not change to "we
>>>> release when we're sure there is definitely improvement in the tree
>>>> compared to last release"?
>>>
>>> The current guideline is quite objective, if there are no reported
>>> bugs and osstest flight doesn't show any regressions we are ready to
>>> release. OTOH how should the improvements to the tree be quantized and
>>> measured?
>>
>> Say, a security bug is fixed? A major bug is closed?
> 
> I think this is still quite subjective, whereas the previous criteria
> was objective.
> 
> Who will take the decision of whether a bug is major or not?
> 
>>>
>>> At any point during the development or the release process the tree
>>> will contain improvements in some areas compared to the last
>>> release.
>>
>> Yes, that is right. That's what CD does, right?
> 
> I thin so, but I'm not an expert on development techniques TBH :).
> 
> IMO one of the problems with Xen is that users don't tend to test
> master often, I assume this is because Xen is a critical piece of
> their infra, and they require it to be completely stable. Not everyone
> can effort an extra box just for testing Xen master. I'm not sure this
> is going to change a lot even if nightly builds are provided.

Since i actually do (on my homeserver), just to share the experience:
- Master/xen-unstable is actually quite stable !
- Most issues i encounter are boot-issues and not uncommonly boot issues
  due to upstream linux kernel changes (by other developers) which have
  an unforeseen impact on Xen.
- So one of the issues with testing is other projects where Xen depends
  upon, and which are out of your control.
- But that's n=1 and on older hardware, so i don't run into issue
  with newer hardware-features.

One thing i haven't seen being mentioned regarding OSSTEST and testing
in general is the perhaps a bit unwieldy test matrix. For a lot of Xen
functionality there are at least 2 options.
I do understand that feature deprecation isn't an easy thing to do and
there are always reasons to keep stuff around and even if you do it, it
still takes time to reap the benefits, but the benefits could be:
    - (a lot of) code cleanup, which eases development in general.
    - much less building and testing that has to be done.
    - easier documentation wise.
    - in the end, easier for users as well.

But perhaps it won't hurt to have some discussion about whether, *why*
and until when, certain features/sub-systems are still worth to keep and
for how long (for example: qemu-trad <-> qemu-xen, PV <-> PVH, seabios
<-> rombios). Since deprecation generally takes a few releases i think
it could be wise to have a discussion about it upfront of every release
so an deprecation warning can be incorporated in that release and the
release notes if need be.

--
Sander

> This is different from say an email or IRC clients, where people don't
> mind that much using unstable versions, and so the development branch
> gets more testing even before the release process starts.
> 
> Roger.
> 
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel