Re: [Xen-devel] [PATCH for-4.9] livepatch: Declare live patching as a supported feature

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 26/06/17 18:30, Andrew Cooper wrote:
> On 26/06/17 18:00, George Dunlap wrote:
>> On 26/06/17 16:36, Ross Lagerwall wrote:
>>> Xen Live Patching has been available as tech preview feature since Xen
>>> 4.7 and has now had a couple of releases to stabilize. Xen Live patching
>>> has been used by multiple vendors to fix several real-world security
>>> issues without any severe bugs encountered. Additionally, there are now
>>> tests in OSSTest that test live patching to ensure that no regressions
>>> are introduced.
>>>
>>> Based on the amount of testing and usage it has had, we are ready to
>>> declare live patching as a 'Supported' feature.
>> Great write-up, Ross, thanks.  I more or less agree with everything
>> except...
>>
>>> * Bugs in livepatch-build-tools creating incorrect live patch that
>>>   results in an insecure host:
>>>     If livepatch-build-tools creates an incorrect live patch that
>>>     results in an insecure host, this shall not be considered a security
>>>     issue. There are too many OSes and toolchains to consider supporting
>>>     this. A live patch should be checked to verify that it is valid
>>>     before loading.
>> I'm not sure I follow the argument here.  Suppose in one months' time it
>> is discovered that livepatch-build-tools, under some circumstances,
>> creates patches that open up a side vulnerability.  Do you really think
>> we should just post a fix to the mailing list, without alerting anybody
>> who may be affected by it?
> 
> There are a million ways this could happen, starting from the simple
> cases of accidentally building the livepatch from a non-clean working
> tree, or accidentally using a compiler other than the one used to build
> the running hypervisor.
> 
> We absolutely cannot be in the position of issuing XSAs for situations
> like this, because there are too many ways where it definitely will go
> wrong, and we'd end up issuing XSAs saying "remember to clean your
> working tree before building a livepatch".  This is of course absurd.

Your argument is that because we do not issue XSAs for *user mistakes*,
that therefore we should not issue XSAs for *bugs in the tool*.

That is of course absurd.  We do not issue XSAs for user mistakes in
building the hypervisor either (for instance, switching gcc versions
without cleaning the hypervisor tree), and yet we still issue XSAs for
bugs in the hypervisor itself.

> IMO, The only viable option is to exclude livepatch-build-tools entirely
> from security scope.  It is already the case that people producing
> livepatches need to check the resulting livepatch binary for sanity, and
> test it suitably in a development environment before use in production.

Look, it sounds like right now you are going through all the livepatches
with a fine-tooth comb *because* the tools are (or recently have been)
unreliable.  But at some point in the future, the patch generation
mechanism will become more reliable.  After 20 XSAs over six months in
which the livepatch tool created the correct patch, you will become more
complacent.  You won't look as closely; it's human nature.

You seem to be simply refusing to use your imagination.  Step back.
Imagine yourself in one year.  You come to the office and find an e-mail
on security@ which says, "Livepatch tools open a security hole when
compiling with gcc x.yy".  You realize that XenVerson ${LATEST-2} uses
gcc x.yy, so you take a closer look at that livepatch, only to discover
that the livepatches generated actually do contain the bug, but you
missed it because ${LATEST-[0,1]} were perfectly fine (since they used
newer versions of gcc), the difference was subtle, and it passed all the
functional tests.

Now all of the customers that have applied those patches are vulnerable.

Do you:

1. Tell the reporter to post it publicly to xen-devel immediately, since
livepatch tools are not security supported -- thus "zero-day"-ing all
your customers (as well as anyone else who happens to have used x.yy to
build a hypervisor)?

2. Secretly take advantage of Citrix' privileged position on the
security list, and try to get an update out to your customers before it
gets announced (but allowing everyone *else* using gcc x.yy to
experience a zero-day)?

3. Issue an XSA so that everyone has the opportunity to fix things up
before making a public announcement, and so that anyone not on the
embargo list gets an alert, so they know to either update their own
livepatches, or look for updates from their software provider?

I think #3 is the only possible choice.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel