Re: [Xen-devel] Is:livepatch-build-tools.git declare it supported? Was:Re: [PATCH for-4.9] livepatch: Declare live patching as a supported feature

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 08/21/2017 01:07 PM, Jan Beulich wrote:
>>>> And remember, this is not "We have tested all compiler versions and
>>>> promise you there are no bugs."  It's, "If someone finds a bug for this
>>>> set of compilers, we will tell you about it so you can do something
>>>> about it."
>>>
>>> I can see and understand all of what you say; my argument,
>>> however was more towards the matrix of what needs supporting
>>> possibly becoming unreasonably large (no matter whether we
>>> specify a range of compilers, as once again distros tend to not
>>> ship plain unpatched upstream compiler versions).
>>
>> What do you mean, "The matrix of what needs supporting [may possibly
>> become] increasingly large"?   What is the problem with having a large
>> (implicit) "supported" matrix?  How is supporting a "large matrix" for
>> livepatch tools different than the current "large matrix" we support
>> for just building Xen at all?
> 
> The matrix of Xen only has just a single dimension. Since livepatch
> tools and Xen are independent, any pair of them would need
> building/testing in order to be sure things work in all supported
> combinations.

So your argument seems to be:

1. We can only provide security support in situations where we can test
all possible combinations in the support matrix.

2. We cannot test the entire matrix of combinations for Xen x livepatch
tools x compilers

3. Therefore, we cannot provide security support for livepatching tools.

Put this way, I hope you can see what the flaw in the argument is: #1 is
false.  Xen has {Xen version} x {Linux version} x {Compiler} x
{Hardware}.  Hardware of course includes not only the chip itself, but
the BIOS / firmware, and the particular devices (and device firmware).
If we wanted we could add in {Python version} for people using pygrub,
and {Ocaml compiler version} for people running Ocaml, versions of
systemd -- I'm sure with effort I could find more dimensions to add to
the matrix.

We do not, and never have, *tested* the entire matrix of possible
combinations considered "security supported" to make sure they work.
Such a matrix is completely impossible to even consider, and even if we
did some sort of testing, that could not guarantee that they are bug free.

What we do for security support is:

1. Test a *representative sample* of combinations (via osstest, product
testing, user testing, &c)

2. Promise to issue XSAs if anyone *happens to discover* a combination
in the rest of the support matrix that has a security issue

That is the requirment for normal Xen, and it would be the same
requirement for livepatch-tools: That between osstest, product, and the
community, we get regular testing of *a representative sample* of {Xen,
livepatch-tools, compiler}, and (what primarily concerns me) issue an
XSA if anyone discovers a security issue somewhere in that matrix.

I'm not frustrated, but I am baffled by the fact that this "support
matrix" objection is so persistent.  Nearly everyone has brought it up,
as though "test every combination" was a necessary requirement, in spite
of the fact that 1) there is *no* piece of software for which we test
the entire matrix of possible combinations 2) I have said over and over
again (in fact, I specifically said a few replies ago -- it's there at
the top of this email) that we do not test all possible combinations.

>> I have elsewhere described a hypothetical scenario where I think we
>> should issue an XSA for livepatch-tools.  Are you really seriously
>> suggesting that in that scenario we should simply publish the
>> vulnerability onto xen-devel with no predisclosure?
> 
> Well, at least I'm not 100% convinced issuing an XSA in this case
> would be appropriate.
> 
> Anyway - since it feels like we're moving in circles (which in part
> may be because I can't express well enough the reasons for my
> hesitation to go to the full XSA extent with the livepatch tools)
> I'd like to just conclude my part here with saying that I'm not
> going to stand in the way whichever decision is taken. I've
> voiced my reservations, and that will have to do. I'd therefore
> prefer to leave the discussion to those more familiar with those
> tools (and their possible limitations and issues).

Indeed; and as I think I said before, I think we need to move forward
with getting a statement on livepatching in, and since most of the
voices involved in this conversation seem to be in favor of saying
livepatch-tools are *not* supported, I won't object. I'm only still
continuing this thread because people seem to be confused about what I
am asking people to do.

I think the likelihood of an XSA-worthy bug being found in the livepatch
tools is very low.  I'm happy to defer the argument about whether we
should issue an XSA for such a bug until such time as one becomes known.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel