[Xen-devel] Is:livepatch-build-tools.git declare it supported? Was:Re: [PATCH for-4.9] livepatch: Declare live patching as a supported feature

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Thu, Aug 03, 2017 at 06:21:30PM +0100, George Dunlap wrote:
> On 08/03/2017 06:20 PM, George Dunlap wrote:
> > On 07/03/2017 03:53 PM, Ross Lagerwall wrote:
> >> On 06/30/2017 02:42 PM, George Dunlap wrote:
> >>> On 06/28/2017 05:18 PM, Ross Lagerwall wrote:
> >>>> On 06/27/2017 10:17 AM, George Dunlap wrote:
> >>>>> On 26/06/17 18:30, Andrew Cooper wrote:
> >>>>>> On 26/06/17 18:00, George Dunlap wrote:
> >>>>>>> On 26/06/17 16:36, Ross Lagerwall wrote:
> >>>> ...
> >>>>> You seem to be simply refusing to use your imagination.  Step back.
> >>>>> Imagine yourself in one year.  You come to the office and find an
> >>>>> e-mail
> >>>>> on security@ which says, "Livepatch tools open a security hole when
> >>>>> compiling with gcc x.yy".  You realize that XenVerson ${LATEST-2} uses
> >>>>> gcc x.yy, so you take a closer look at that livepatch, only to discover
> >>>>> that the livepatches generated actually do contain the bug, but you
> >>>>> missed it because ${LATEST-[0,1]} were perfectly fine (since they used
> >>>>> newer versions of gcc), the difference was subtle, and it passed all
> >>>>> the
> >>>>> functional tests.
> >>>>>
> >>>>> Now all of the customers that have applied those patches are
> >>>>> vulnerable.
> >>>>>
> >>>>> Do you:
> >>>>>
> >>>>> 1. Tell the reporter to post it publicly to xen-devel immediately,
> >>>>> since
> >>>>> livepatch tools are not security supported -- thus "zero-day"-ing all
> >>>>> your customers (as well as anyone else who happens to have used x.yy to
> >>>>> build a hypervisor)?
> >>>>>
> >>>>> 2. Secretly take advantage of Citrix' privileged position on the
> >>>>> security list, and try to get an update out to your customers before it
> >>>>> gets announced (but allowing everyone *else* using gcc x.yy to
> >>>>> experience a zero-day)?
> >>>>>
> >>>>> 3. Issue an XSA so that everyone has the opportunity to fix things up
> >>>>> before making a public announcement, and so that anyone not on the
> >>>>> embargo list gets an alert, so they know to either update their own
> >>>>> livepatches, or look for updates from their software provider?
> >>>>>
> >>>>> I think #3 is the only possible choice.
> >>>>>
> >>>>>    -George
> >>>>>
> >>>>
> >>>> The issue here is that any bug in livepatch-build-tools which still
> >>>> results in output being generated would be a security issue, because
> >>>> someone might have used it to patch a security issue.
> >>>> livepatch-build-tools is certainly not stable enough yet (ever?) to be
> >>>> treated in this fashion.
> >>>
> >>> You didn't answer my question.  If the situation described happens, what
> >>> position do you want Andrew to be put in?  (If I missed a potential
> >>> action, let me know.)
> >>>
> >>
> >> I would choose #3 as it is the obvious choice. But I still don't think
> >> it is a sensible idea to have security support for the build tools, at
> >> least at this point. The same scenario could be posed for a nasty bug
> >> that affects Xen 4.4 only, but it is now just out of security support.
> >> IMO something being not supported doesn't preclude it from having an XSA
> >> released if there is a particularly nasty vulnerability found.
> > 
> > Well basically I think we agree, but we're using different terms.  You
> > want to say, "This isn't security supported, but if important bug is
> > actually found then we'll issue an XSA".  I want to say, "This is
> > security supported, because if an important bug is actually found we'll
> > issue an XSA."
> > 
> > So it seems to me there are likely two things that make you resistant to
> > calling it "security supported":
> > 
> > 1. The fear that we'll be issuing XSAs over trivial things that don't matter
> > 
> > 2. The fear that people will not do due diligence when creating patches
> > with the tools.
> > 
> > I think #1 is just a misconception.  *Every* bug reported to us about
> > any part of the code we go through the process of trying to determine
> > its impact and whether we need to issue an XSA or not.  All of the
> > examples put forward of things we don't want to issue an XSA for are
> > things that I'm sure we would not issue an XSA for.
> > 
> > For #2, that is a reasonable fear, but we can deal with that in a
> > different way than calling the tools "unsupported".  We can, for
> > instance, mention that in the documents.  We can add a warning message
> > that the build tools output saying that the result should be manually
> > inspected for correctness.
> 
> We need to get a resolution on this.  Anyone else (particarly
> committers) want to give their opinion?

Changing title as this is all about now livepatch-build-tools.

The livepatch-build-tools get a lot of usage around XSA times.
And that is when the corner cases are being found. The three of them:
0c10457 Remove section alignment requirement
b30d34c Ignore .discard sections
6327ab9 create-diff-object: Update fixup offsets in .rela.ex_table

where thanks to generating XSAs. Now the folks who use these tools
are also the ones that do pre-disclosures. And the folks who
work on these tools also are the ones who have to get the livepatches out.

It is a stressful time and in the past the issues were off:
'oh, livepatch-build-tools won't generate the livepatch' 

which I don't even know how to classify - is it an XSA that it could not
create an livepatch?

And if the livepatch-build-tools does generate something mighty wrong
then the folks on the XSA pre-disclosure list should be let know
(and that has been happening).

But I am not really a fan of 'Oh, and one more XSA'


The second argument is that livepatch-build-tools is like the GCC compiler.
In fact it takes the binary blob of what the compiler has produced
and checks it against the other one. If the compiler adds extra instructions
or changes the instructions slightly we will classify that as
code needing to be patched (and yes that has come up).

This is very similar to what XSA-155 was - the GCC compiler optimizations
added a nice jump table that was accessed twice. And the offset was
retrieved from the shared ring.

But we didn't do an XSA-155 for the GCC compiler. That is we didn't
file a ticket with GCC saying 'Hey, your compiler can create an race
on shared memory. Could you make your compiler be smarter in these cases'
We instead wrote code with this optimization in mind with more
barriers.

I think livepatch-build-tools is in the same category as GCC or linkers.
> 
>  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel