[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Notes Design Session: Making Releases Lessons Learned: Improving Our Release Process and Tooling

Hi all,
these are my notes from http://sched.co/AjES
These are a little crude and I may have missed some things or misrepresented

* Wei Liu
* Julien Grall
* Lars Kurth
* Ian Jackson

Feel free to correct

Releases Lessons learned

Process issues in 4.9.0 (checklists, guides, templates)

* Lots of fiddly bits need to be executed (right now we have a release 
technician checklist)
* We don't have a comprehensive checklist that is useful for Release Managers
* Started the final steps for 4.9.0 on Tue of the release week
* By this tine we were committed to releasing on Wed (no point of return due to 
* Some items were release managers job, but discovered by Release Technician 
* Release technician checklist is in xen.git

ACTION (Wei, Julien, Lars): Have a how to be release manager file (needs to 
contain some of the stuff in Release technician checklist)
 * RM file: clear set of criteria on PR. Go/NoGo part
 * See 
for a starting point

ACTION (Lars/Wei/Julien) to write "standard e-mail templates for common stuff" 
rather than re-doing these every single time
ACTION (Ian): Clean up release technician checklist after we have the how to be 
release manager file

ACTION (Wei/Julien): Additional stuff to add to the templates/RM guide
* Add hand-over of tasks for Release Manager responsibility to the "how to be 
release manager" file
* Add clear reminders in particular at the beginning of a release into e-mail 
templates: such as "put dates X,Y, Z in your calendar"
* Communicate better when tree is open again (add to checklist and templates)

ARM issues in 4.9.0

Julien: Unreliable testing on ARM32/64 
- ongoing problems with ARM64 box (hardware issue), other ARM64 box (firmware 
issue), ARM32 Arndale hardware problem (replace)

On ARM everything is in progress and not much more can be done => no concrete 
ACTION as those already executed

x86 Heisenbugs in 4.9.0
- x86 migration Heisenbugs have caused us issues this release cycle: not 
completely understood the issues and hard to debug

Ian: These are most likely software problems, most likely in Xen.
ISSUE: nobody wants to debug Windows Heisenbugs
ISSUE: have nobody to do formal triage - doing back-pressure on developers

ACTION: Ian, Julien
- Manually add ignore list for specific Heisenbugs until we found another 
- Requires a proposal to the list - Julien to propose

ACTION: Julien to propose 
- OSSTEST to CC people blocked/proposed on test failures instead just the list
- Release manager can say "not releasing now" because of too many bugs
  "until someone fixes these". "no more patches until XYZ" 

Improving the Process

Open source projects do NOT need licenses for JIRA: these was raised by OpenXT 
folks, who use JIRA

ACTION: Lars to follow up with Atlassian - 
ACTION: Lars to follow up with Intel on owner and describe simple process to 
nominate new people

We then had a brief discussion on whether to use labels or components. OpenXT 
believe components are easier to manage

ACTION: Julien - Decide on whether to use labels or components for wiki page
ACTION: Julien - Make a proposal on components on list for review (note: Lars 
could help based on 

XSA Testing 
ACTION: include Lars'es tool into checklist

Arguments on the mailing list on XSAs
Delayed RC by two days: security patches produced by the security team - we 
don't always consult the maintainer

ACTION: committers@ + maintainer of relevant code of Advisory should be on 
pre-disclosure list 
* Explicitly say in public that security team should be informed pre-disclosure 
  With appropriate template

Resolved in 

Release Manager for 4.11
Julien stepping down after 4.10 - we should find another volunteer

ACTION: Lars to sound out SUSE (Juergen Gross), Ross Phillipson (Oracle)
Note: Juergen would be happy to do this for one release

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.