Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Lars Kurth <lars.kurth@xxxxxxxxxx>]

Hi all, (I also moved the AB to BCC)

I summarized the discussion in 
https://docs.google.com/document/d/1W7OuISUau-FtPG6tIinD4GXYFb-hKDjaqTj84pogNrA/edit?usp=sharing
 

I may have missed some things or misinterpreted them, but it looks as if 
consensus is emerging in some areas. I would like to discuss what we do for the 
4.12 release at next week's community call. As far as I can see we have a few 
options:
* Go on as we are
* Move to 9 months, until we fixed the underlying issues - the problem is that 
unless we get some sort of commitment 
* Skip a release as a one-off: Set ourselves some goals that must be achieved 
in this cycle around testing - this will need some commitment from vendors

Regards
Lars

On 06/07/2018, 16:09, "Ian Jackson" <ian.jackson@xxxxxxxxxx> wrote:

    Doug Goldstein writes ("Re: [Xen-devel] [Notes for xen summit 2018 design 
session] Process changes: is the 6 monthly release Cadence too short, Security 
Process, ..."):
    > You effectively supported my point in the end. People value their time.
    > Giving them details about trends could help folks to look at test
    > failures that have "interesting" failures.
    
    Your focuse on the notion of "trend" here is interesting.
    
    One way of looking at that is that really you are asking about change
    over time.  Of course that's what the "regression" concept is in
    osstest.
    
    But that is for each individual test step.  When we have a heisenbug
    which affects multiple tests, osstest is not really right now very
    good at aggregating that information.
    
    Maybe your "trend" idea is useful here.  osstest could perhaps track
    the proportion of "heisen" failures somehow.  I will have to think
    about how to do that.  Ie, exactly how to calculate the numerator and
    denominator.
    
    Do we want to track that over flights that got pushes, or something,
    only ?  ISTM that, for example, if master==staging~2, and tests of
    staging~1 were a wipeout because of a regression fixed in staging~0,
    then when we report the "trend" in the test report of staging~0, that
    should disregard the disaster that was the test(s) of staging~1.
    
    And of course I should be asking a different question entirely if we
    decide to move to multiple, rewinding, input branches, rather than a
    single fast-forwarding one.
    
    Ian.
    

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel