Re: [Xen-devel] [Notes for xen summit 2018 design session] Process changes: is the 6 monthly release Cadence too short, Security Process, ...

[Lars Kurth <lars.kurth@xxxxxxxxxx>]

Hi all,
at some point we need to take a step back and summarize this discussion and 
establish 
a) what seems to be agreed
b) what is possible
c) and what is controversial
I am sort of volunteering for this and was planning to do so tomorrow. 
Lars

On 05/07/2018, 13:20, "Juergen Gross" <jgross@xxxxxxxx> wrote:

    On 05/07/18 13:51, Andrew Cooper wrote:
    > On 05/07/18 12:18, George Dunlap wrote:
    >>
    >>>>    Another potential problems showed up last week: OSSTEST is using the
    >>>>    Debian servers for doing the basic installation. A change there 
(e.g.
    >>>>    a new point release) will block tests. I'd prefer to have a local 
cache
    >>>>    of the last known good set of *.deb files to be used especially for 
the
    >>>>    branched Xen versions. This would rule out remote problems for 
releases.
    >>>>
    >>>> This is again something which we should definitely look at.
    >>> This was bad luck.  This kind of update happens about 3-4 times a
    >>> year.  It does break everything, leading to a delay of a day or two,
    >>> but the fix is straightforward.
    >>>
    >>> Obviously this is not ideal but the solutions are nontrivial.  It is
    >>> not really possible to "have a local cache of the last known good set
    >>> of *.deb files" without knowing what that subset should be; that would
    >>> require an edifice to track what is used, or some manual configuration
    >>> which would probably break.  Alternatively we could run a complete
    >>> mirror but that is a *lot* of space and bandwidth, most of which would
    >>> be unused.
    >>>
    >>> I think the right approach is probably to switch from using d-i for
    >>> host installs, to something like FAI.  That would be faster as well.
    >>> However that amouns to reengineering the way osstest does host
    >>> installs; it would also leave us maintaining an additional way to do
    >>> host installs, since we would still want to be able to *test* d-i
    >>> operation as a guest.
    >> What I think would be ideal is a way to take ‘snapshots’ of different 
states of setup for various hosts and revert to them.  There’s absolutely no 
reason to do a full install of a host every osstest run, when that install 
happens 1) before we even install Xen, and 2) should be nearly identical each 
time.  We should be able to install a host, take a snapshot of the “clean” 
install, then do the build prep, take a snapshot of that, and then simply 
revert to one or both of those (assuming build requirements haven’t changed in 
the mean time) whenever necessary.  Re-generating these snapshots once per week 
per host should be plenty, and sounds like it would massively improve the 
current throughput.
    >>
    >> I’d like to propose the idea also that we try to find a more efficient 
way of testing guest functionality than doing a guest install.  I understand 
it’s a natural way to test a reasonable range of functionality, but 
particularly for Windows guests, my impression is that it’s very slow; there 
must be a way to make a test that would have similar coverage but be able to be 
completed with a pre-installed snapshot, in only a few minutes.
    > 
    > We've had similar discussions in XenServer. That idea is superficially
    > attractive but actually makes things worse, because it now means that
    > filesystem clone/snapshot is now in the mix of things which can go wrong.
    > 
    > Particularly with OSSTest testing mainline kernels, rather than distro
    > stable kernels, the chances of finding filesystem bugs grows
    > substantially, and the complexity of diagnosing an issue is outside of
    > our area of expertise.
    
    But are really all tests required to use a freshly installed mainline
    kernel? Can't we e.g. do a guest install test once a new kernel is
    released and clone the resulting guest disk image for other tests
    (after shutting down the guest, of course)?
    
    Same applies to the host: the base system (without the to be tested
    component like qemu, xen, or whatever) could be installed just by
    cloning a disk/partition/logical volume.
    
    Each image would run through the stages new->staging->stable:
    
    - Each time a component is released an image is based on (e.g. a new
      mainline kernel) a new image is created by installing it. In case this
      succeeds, the image is moved to the staging area.
    - The images in the staging area are tested using known stable
      components in order to test the image, not the test-components. In
      case of all tests succeeded the image is moved to the stable area.
    - The stable images are used to test components from staging. In case of
      success the related components can be pushed to stable/master.
    
    
    Juergen
    

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel