[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] A document for Xen release management

Hi all,

A1 ACTION to write "standard e-mail templates for common stuff", rather than re-doing these every single time
Ian release manager file 

A2 ACTION : Clean up release technician checklist after we have the how to be
* Add hand-over of tasks for Release Manager responsibility to the "how to be release manager file"

A3 ACTION: Additional stuff to add to the templates/RM guide
A3.1: Add clear reminders in particular at the beginning of a release into e-mail templates: such as put dates X,Y, Z in your calendar add to checklist and templates
A3.2: Communicate better when tree is open again
A3.3: Release manager can say "not releasing now" because of too many bugs, "until someone fixes these". "no more patches until XYZ"

Looking through it, I am not sure whether A3.2 and 3.3 are covered.


On 17 Jul 2017, at 16:09, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:

It is agreed during the summit we should write down such document. Here
is my attempt of doing so.

We should probably commit something like this into xen.git so that it
gets updated regularly.

Comments are welcome.


% Xen Release Management
% Wei Liu <<wei.liu2@xxxxxxxxxx>>
% Revision 1

# Motivation

Over the years we have had different people from different company signning
up as the Release Manager of Xen. It would be rather wasteful if every new
Release Manager has to go over everything and tripped over by the same
mistakes again and again.

This file intends to document the process of managing a Xen release. It is
mainly written for Release Manager, but other roles (contributors,
maintainers and committers) are also encouraged to read this document, so
that they can have an idea what to expect from the Release Manager.

# Xen release cycle

The Xen hypervisor project now releases twice a year, at the beginning of
June and the beginning of December. The actual release date depends on a lot
of factors.

We can roughly divide one release into two periods. The development period
and the freeze period. The former is 4 months long and the latter is about 2
months long.

During development period, contributors submit patches to be reviewed and
committed into xen.git.

During freeze period, the tree is closed for new features. Only bug fixes are
accepted. This period can be shorter or longer than 2 months. If it ends up
longer than 2 months, it eats into the next development period.

# The different roles in a Xen release

## Release Manager

A trusted developer in the community that owns the release process. The major
goal of the Release Manager is to make sure a Xen release has high quality
and doens't slip too much.

The Release Manager will not see much workload during development period, but
expects to see increasing workload during the freeze period until the final
release. He or she is expected to keep track of issues, arrange RCs,
negotiate with relevant stakeholders, balance the need from various parties
and make difficult decisions when necessary.

The Release Manager essentially owns xen-unstable branch during the freeze
period. The committers will act on the wishes of the Release Manager during
that time.

## Maintainers

A group of trusted developers who are responsible for certain components in
xen.git. They are expected to respond to patches / questions with regard to
their components in a timely manner, especially during the freeze period.

## Committers

A group of trusted maintainers who can commit to xen.git. During the
development window they normally push things as they see fit. During the
freeze period they transfer xen-unstable branch ownership and act on the
wishes of the Release Manager. That normally means they need to have an
Release Ack in order to push a patch.

## Contributors

Contributors are also expected to respond quickly to any issues regarding the
code they submitted during development period. Failing that, the Release
Manager might decide to revert the changes, declare feature unsupported or
take any action he / she deems appropriate.

## The Security Team

The Security Team operates independently. The visibility might be rather
limited due to the sensitive nature of security work. The best action the
Release Manager can take is to set aside some time for potential security
issues to be fixed.

## The Release Technician

The Release Technician is the person who tags various trees, prepares tarball
etc. He or she acts on the wishes of the Release Manager. Please make sure
the communication is as clear as it can be.

## The Community Manager

The Community Manager owns xenproject.org infrastructure. He or she is
responsible for updating various web archives, updating wiki pages and
coordinating with the PR Personnel.

## The PR Personnel

They are responsible for corrdinating with external reporters to publish Xen
release announcement. The Release Manager should be absolutely sure the
release is going out on a particular date before giving them the signal to
proceed, because there is a point of no return once they schedule a date with
external reporters.

# What happens during a release

## Development period

Send out monthly update email. The email contains the timeline of the
release, the major work items and any other information the Release Manager
sees fit. Please consider adding a recurring event to your calendar.

Occasionally check the status of the xen-unstable branch, make sure it gets
timely pushes to master.

## Freeze period

Before or at very early stage of the freeze period, agree with the Community
Manager a schedule for RC test days.

Once the freeze starts, the ownership of xen-unstable branch automatically
transfers to the Release Manager.

Here is a list of things to do for making RCs:

1. Check the status of the tree. Ask the Release Technician to make an RC if the tree is good.

1. Send an email to xen-devel, xen-users and xen-announce to announce the RC.

1. Branch and / or reopen the tree for further feature submission if appropriate.

1. Collect and track any issues reported, determine their severity, prod relevant developers and maintainers to fix the issues.

1. When patches to fix issues are posted, determine if the patches are good to be included.

1. Go back to 1.

It is normally OK in the early RCs that you hand back xen-unstable branch to
committers so that they can commit bug fixes at will. As we approach late
RCs, the standard for accepting a patch will get higher and higher. Please
communicate clearly when committers can commit at will and when formal
Release Ack is needed.

At the same time, work with the Community Manager, PR Personnel and
Contributors to gather a list of features for the release. Discuss the
support status of new features with stakeholders. Help prepare the press
release, write a blog post for the release.

Does it make sense to move this into a separate section, or have a separate section which list the key steps? If so, I am happy to pull this together. Primarily I tend to drive the PR angle with Zibby and would be happy to create a checklist. The Release Manager's role here is one of providing input, but can (if desired) be more high profile (e.g. quotes in releases). 

When you think all pending issues are fixed and Xen is ready to be released
from the last RC:

1. Send out commit moratorium emails to committers@.

1. Check all the trees (mini-os, qemu-trad, qemu-xen, seabios, ovmf etc).
They have the correct commits and all security patches applied. There will be
tools provided.

1. Ask the Community Manager and Release Technician to double-check all
security patches have been applied. If not, apply them, arrange another RC
and restart this checklist.

I think double checking is good. If http://xenbits.xenproject.org/gitweb/?p=people/larsk/xen-release-scripts.git are deemed to be fit for purpose, we should probably refer to these

1. Ask the Release Technician to tag the trees and make the tarball. Ask the
Community Manager to update relevant web assets.


1. Check with relevant stake-holders (typically community manager) whether wiki documentation and PR is in good shape (for an example see https://wiki.xenproject.org/wiki/Category:Xen_4.9)

1. Give the PR Personnel signal to proceed. Cooridinate with him / her on the
public annoucement.

Typically we will need a bit of lead-time here to ensure that everything is in place

1. Make the announcement on various mailing list, publish the blog post.

Allow for contigencies. It is not uncommon that some last minute (security or
not) bugs are discovered. To provide a fix takes time, the test of the fix
will also take time. Allow for at least 1 week from getting a fix to getting
a push. For security bugs, corrdinate with the Security Team to adjust the
dates according to our security policy.

There should probably be a section along the lines of (for A2)

## Hand over of Release Manager Responsibility

Probably this is an area where Wei, George, Konrad and Julien have experience.

This should include a list of systems a Release Manager should be signed up to, such as blog account, xen-announce, ...

# Email templates

## RC emails

Hi all,

Xen X.Y rcZ is tagged. You can check that out from xen.git:

git://xenbits.xen.org/xen.git X.Y.0-rcZ

For your convenience there is also a tarball at:

And the signature is at:

Please send bug reports and test reports to xen-devel@xxxxxxxxxxxxxxxxxxxx.
When sending bug reports, please CC relevant maintainers and me

As a reminder, there will be another Xen Test Day.

See instructions on: URL_TO_TEST_INSTRUCTIONS

We should probably have mail templates for the specific stages of the process, which can then include reminders to add calendar entries (see A3.1)

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.