Re: [Xen-devel] Security vulnerability process, and CVE-2012-0217

[Matt Wilson <msw@xxxxxxxxxx>]

On Tue, Jun 19, 2012 at 11:16:05AM -0700, Ian Jackson wrote:
> [...]

Thank you for writing all this up, and for all of your efforts as part
of Xen.org and the security response team, Ian. You have already
covered many of my concerns, such as avoiding holidays.

> The big issues
> --------------
>
> 1. Purpose of the process
>
> The first point is that we think the security vulnerability process
> document should have an explanation of what its goals are.  This would
> have greatly assisted us when making some of the more difficult
> decisions.

The security vulnerability process document should most definitely
encapsulate both explicit guidance and broad tenets that can be used
to make tough calls. I think that these should be explicitly called
out in front matter as an evolutionary part of the process. Tenets
should always be open to being refined or redefined as Xen.org
projects grow and usage shifts.

> In particular, if the process explained its purpose, we would be able
> to refer to the spirit of the document when making interpretation
> decisions or dealing with issues not clearly resolved by the document.
>
> In this context we need to decide whether:
>  (a) the predisclosure arrangements are just there so that
>      organisations can backport patches, develop and test updated
>      versions, and so forth;
>  (b) the arrangements are also intended to allow operators to deploy
>      fixed versions.

I think that there may be other aspects of the predisclosure period
that deserve explicit callouts, so I don't think that it's helpful to
frame this discussion in particular context that only gives us choices
(a) or (b). For example, we might want to explicitly call out the
purpose of the predisclosure period to include further testing and
validation of a fix from a group more broad than the security response
team.

But taking a step back, I propose that a core tenet of the security
response process should be to reduce days-of-risk for all consumers of
Xen.org projects, whether direct or indirect, to zero. Days-of-risk
can be fuzzy to measure, so we could define this as the number of days
between when a security problem is publicly known (e.g. through
evidence of exploitation in the wild or a public announcement) and
when the problem has been addressed such that there is no longer any
risk (e.g., through a Xen consumer deploying a fixed version from a
vendor or an infrastructure provider doing the same on a consumerâs
behalf.)

This measure might not match up perfectly with typical days-of-risk
assessments that some software vendors make. For example, Red Hat
measures days-of-risk from the date of public disclosure to the date
that a fixed package is made available. They publish metrics on this,
e.g.:
  http://www.redhat.com/security/data/metrics/summary-rhel6-critical.html

> Of course this needs to deal clearly with the common stituation of an
> organisation running Xen which is a direct consumer of Xen.org source
> code.

Indeed. If a goal of the process is to reduce days-of-risk for all
consumers of Xen.org projects to zero, coordinating package updates
only from software vendors will not achieve it.

[...]
>
> 2. Extension of embargo dates
>
> The most controversial decision was whether the embargo date might be
> extended after it had originally been set.  We resisted these
> suggestions, referring to the process document, which does not
> contemplate extending the disclosure period.  However, when a
> predisclosure list member pressured the discoverer into requesting an
> extension, we felt the best interpretation of the process document
> required us to acquiesce.
>
> Specifically, of course, we as a team would like clearer guidance from
> the process about whether, and if so under what circumstances, a
> predisclosure period should be extended.

I think that the issue of establishing and changing the embargo end
date, whether extending or shorting, deserves a lot of discussion.
It's good that we're not limiting the discussion only to extending
extending disclosure dates, but also establishing the default timeline
as you call out in 4 below.  I think that some guidelines for
establishing an appropriate timeline can address problems that cause
extension requests.

The default disclosure period, if agreeable to the discoverer, is
currently three weeks:
   1. One working week between notification arriving at security@xen
      and the issue of our own advisory to our predisclosure list. We
      will use this time to gather information and prepare our
      advisory, including required patches.
   2. Two working weeks between issue of our advisory to our
      predisclosure list and publication.

This may be a reasonable starting point for general issues, but I
think that the overall impact and complexity of an issue needs to be
considered when establishing a timeline.

For example, the oCERT Disclosure Policy gives this guidance:
  https://www.ocert.org/disclosure_policy.html
"""
   The following time frames regulate oCERT embargo proposals:

   - 7 days, in case the issue is already well narrowed down and
     tested, requiring trivial configuration and/or code change

   - 14 days, standard embargo for most cases

   - 30 days, in case of critical and complex vulnerabilities
     (example, trivial exploitation of administrative privileges on a
     static library affecting a large number of packages), and with
     the agreement of all parties

   - under extremely exceptional circumstances, if the oCERT Team and
     all the parties involved feel the need for longer time, a 2
     months embargo can be applied, in this case we would clearly
     document the decision for public review

   - in any circumstance reporter preference will always be honoured
     in case a joint agreement is not reached, as oCERT would be
     anyway unable to force its embargo
"""

The CERT process defaults to 45 days, with guidance to shorten the
release schedule if there's evidence of active exploitation:
 http://www.cert.org/kb/vul_disclosure.html
"""
   Q: Will all vulnerabilities be disclosed within 45 days?

   A: No. There may often be circumstances that will cause us to
      adjust our publication schedule. Threats that are especially
      serious or for which we have evidence of exploitation will
      likely cause us to shorten our release schedule. Threats that
      require "hard" changes (changes to standards, changes to core
      operating system components) will cause us to extend our
      publication schedule.
"""

For another take, the WebKit default embargo period, found here:
http://www.webkit.org/security/, is 60 days unless all vendors have
addressed an issue, in which case an embargo may be lifted sooner.

> 3. Decisionmaking
>
> It was suggested to us several times, and indeed seemed to be regarded
> by some as an underlying principle, that the predisclosure list
> members should be making the decisions about disclosure schedule etc.
>
> The question of who is to make the decisions, and on what basis, needs
> to be addressed.

I do not think that it is wise for the policy to establish decision
making by committee. All concerned parties should voice any concerns
regarding the security response process so that the resulting document
will guide those making decisions.

To put it another way, predisclosure list members (and any other
interested party) should make decisions on how they'd like for
disclosure schedules to be established now, and the ratified process
should act as a proxy when specific vulnerabilities are handled.

In the interest of transparency, those making decisions, i.e. the
members of the security response team and any other person or entity
on the security@xxxxxxx alias, should be disclosed and documented in
the process document.

Ultimately I think that, as is common practice in coordinated
vulnerability disclosure, the discoverer is the ultimate
decision-maker as to what information is disclosed and when it is
disclosed. The xen@security team makes decisions per the guidance in
the process and the facts at hand.

> 4. Length of (default) predisclosure period
>
[...]

See comments above.

> 5. Criteria for predisclosure list membership
>
[...]
> We need to clarify whether upstreams and hardware vendors should be on
> the predisclosure list.  Currently we have one notable upstream vendor
> on that list.
>
> Our preliminary view is that the predisclosure list should be used for
> downstream notifications.  Upstreams, including hardware
> manufacturers, should be brought in to the disclosure process as
> needed.  And as noted, our process for making sure we do that properly
> needs to be formalised.

Overall I think that, in a coordinated disclosure response,
information should be distributed on a need-to-know basis.

[...]
> 6. Sharing of information and code between predisclosure participants
>
> We need the process document to be clearer about what kinds of
> communications are contemplated _between_ members of the predisclosure
> list.
>
> One particular issue here which also relates to the predisclosure
> membership criteria, is whether large indirect consumers of Xen should
> be on the predisclosure list in their own right.  That would allow
> them to deploy the fix before the embargo date.  It would also allow
> them to prepare for testing and deployment, before the fix is
> available from their vendor (who would in this scenario also be
> entitled to be a predisclosure list member).
>
> In this context, the process needs to clarify whether vendors may
> release fixed binaries during the predisclosure period.  Certainly
> these should not be released other than to predisclosure list members,
> since the nature of a bug can often by discovered by reverse
> engineering.  But can they be released to predisclousre list members ?

In my opinion, there is less distinction between "software vendors"
and "large indirect consumers of Xen" than you suggest.

> More minor policy questions
> ---------------------------
>
[...]
> 9. Vulnerability process scope
>
> We need to clarify the scope of the process document.  Currently it
> looks like it covers every Xen.org project.
>
> While it would be a nice ideal to support every Xen.org project this
> way, in practice the team behind security@xen do not have the
> expertise or resources to fix problems in (say) XCP, or the ARM PV
> port.  Our capability is concentrated on the "Upstream Xen" codebase
> (xen-*.hg and its sub-repositories) and the Xen support code in Linux.
>
> Perhaps this could be dealt with by making it clear that problems are
> handled on a best effort basis.

It would seem that in cases with mature projects, appropriate
maintainers and committers should be engaged by security@xxxxxxx to
address problems and prepare fixes. security@xxxxxxx should, I think,
continue to coordinate the overall response.

In order to graduate, a project should maintain sufficient sponsorship
to address security problems in a timely manner. That should be part
of the "basic conditions" of being labeled a "mature" project. If a
mature project does not have sufficient sponsorship to address
security concerns, it should be called out and archiving the project
should be considered.

The Apache Incubator process has some good advice to projects that are
graduating: http://incubator.apache.org/guides/graduation.html#security

> 10. Exploits
>
> We need a clear policy about releasing proof of concept exploits -
> whether, when and who to.

Let's not limit the discussion only to PoC exploit code. The policy
should speak to who gets access to specific technical details of a
vulnerability, and when they gain access.

Parties that are *active* in the coordinated disclosure, for example
by building patched versions of products or services built on Xen,
need to have access to as much technical information as possible in
order to validate the fix and their ports. Those that are passive, who
may be deploying a pre-release fixed version from a vendor, do not
need access to these materials.

[...]
>
> Lacunae in the process
> ----------------------
>
> 12. Holidays
>
> The original planned disclosure date, 2012-05-01, was a public holiday
> in many places.  We should try to avoid holidays, and Fridays.  We
> need to discuss which territories' holidays should be checked and make
> a list for inclusion in the process.

This makes sense, and is similar to other coordinated response groups
like linux-distros@xxxxxxxxxxxxxxxx

> 13. Patch development and review
>
> The patch development process is too closed and not robust enough.
>
> We need to be much clearer both about the need for security patches to
> fix things in the smallest, simplest and most reliable way, and not
> fix or "improve" matters in unrelated ways.  Our internal code review
> needs to be much better.
>
> We need to clarify that other issues discovered during the course of
> investigating a security disclosure will not be publicly discussed
> without first consulting the Xen.org security team, regardless of
> their actual relationship to the vulnerability at hand or expected
> security impact.  Otherwise there is a substantial risk that such
> changes will draw attention to embargoed problems.

All of this makes sense to me.

> It may be that it would be a better idea to send out earlier
> advisories without patches to the predisclosure list, to enable those
> predisclosure list members who would like to do so to help with
> development and testing of the patches.  If so this needs to be
> catered for.

This would be tremendously helpful, so long as adequate technical
information is provided.

> 14. Early consideration of which other organisations to bring in
>
> Our process needs to have, very early on, an explicit step to of
> deciding which other projects/organisations may also be vulnerable and
> may therefore need to be part of the same disclosure process.  It also
> needs to make sure that we ask for any help (for example from
> upstreams or hardware vendors) as soon as possible.

The existing process had a step for this:
"""
  d. If we think other software systems (for example, competing
     hypervisor systems) are likely to be affected by the same
     vulnerability, we will try to make those other projects aware of
     the problem and include them in the advisory preparation process.
"""

> [...]

Thank you again for acting as the facilitator for this discussion, and
many thanks to everyone who works tirelessly as part of the Xen.org
community on issues like these.

Matt

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel