Re: [Xen-devel] [PATCH 0/3 for 4.8] docs: feature documents for the schedulers

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 14 Oct 2016, Andrew Cooper wrote:
> > I like the idea of keeping these info on pandoc on a git repo, like Lars
> > did with the governance.
> 
> I should hasten to add that perhaps picking on the security team in
> isolation was a poor move on my part, for which I apologise. There are
> multiple involved parties with different interests, all of which need
> attending to.

No worries, on my part I just think that we should write down these
things to remember, set expectations and to avoid surprises.


> The feature submitter (should) have a vested interest in getting code
> submitted (even as experimental/tech preview to start with), so that
> others can trial/debug/extend the feature.  Even having code upstream
> but disabled-by-default in Kconfig is a far better position for others
> to start from, than to be provided with instructions saying "there was a
> patch series several months ago on a mailing list".

Undeniably true.

 
> The reviewers/maintainers (should) have a vested interest in keeping the
> overall quality up.  Due to their appointment within the community, they
> have demonstrated knowledge and expertise in their respective areas, and
> the overall project relies on them to ask questions such as "How does $X
> work with $Y?" or "Have you considered $Z as an alternative?", and to
> come to a fair and unbiased opinion to the quality and worthiness of a
> feature.
> 
> The release manager has conflicted vested interests.  On the one hand,
> getting more features in a release is better on paper and in the press,
> but can certainly backfire when features aren't up to scratch. 
> Ultimately, if features in a release are substandard, the downstreams
> and end users will be the ones to bear the pain.  It is in the release
> managers best interest to ensure that the features which are finally
> accepted are actually ready.
> 
> Nothing is ever perfect, but a whole lot of stuff is perfectly good in
> common case, and useful for people in general.  This is why the feature
> doc template specifically has sections for know issues to be fixed, and
> areas for future improvement.  Any documentation (so long as it isn't
> factually incorrect) is better than nothing.
> 
> There seems to be general consensus that a status of "Supported" means
> "with security support", and I agree with this assessment.  Ultimately,
> the security team is accountable to whatever the project as a whole
> declares "Supported", but as the security team are all commiters and
> maintainer there is a large overlap of responsibility.  OTOH, the
> security team are also responsible for managing the fallout when things
> go wrong.
> 
> XSAs are bad publicity, and are a problem for downstreams (remember that
> we have plenty of downstreams who measure quantity of servers in units
> of a datacenter).  Another problem we as a community have is that no
> support status is written down; there is a 13ish? year backlog in this
> regard.  Writing details down in feature docs is intended to get
> everyone on the same, un-ambiguous, page.
> 
> We also have an unfortunate habit of new features appearing, being
> accepted, and (intentionally or unintentionally) available to guests. 
> Fastforward a bit, and it is discovered that said feature resembles a
> sieve in terms of security holes, and the common recourse is to publish
> an XSA stating "turn it off and don't use it with untrusted guests", or
> "here is a patch to actually turn it off, and still don't use it with
> untrusted guests", because we really can't re-engineer a feature in a
> security patch.  (Sorry to pick on TMEM here, but it is now 4 years of
> development later and it still has a lot of development left to go. 
> Reworking features in the light of a security hole can be very
> complicated, and lots of work.)

Yes, that's definitely a position we should try not to be in.


> There should be a high barrier to "Supported" status, because the cost
> of getting it wrong is equally high.  However, there are perfectly
> legitimate intermediate stages such as "Supported in these limited set
> of circumstances".  A number of features are not plausibly for use in
> production environments, but otherwise function fine for
> development/investigatory purposes.  In these cases, something like "no
> security support, but believed to be working fine" might be appropriate.

I agree on this. I think we need an intermediate step: "working but not
supported for security" is completely reasonable. When we say that it is
"working", it should be because we have automated testing for it (I
don't know if I would go as far as requiring it to be in OSSTest, any
automated testing, even third party, would do). If it is not
automatically tested, then it is just "best effort".

Finally some features involve some sort of ABI exposed to the guest. At
some point when the feature is "Supported", the ABI will be most
certainly maintained for backward compatibility. But the feature could
be fully working, automatically tested, but still the ABI could be not
stable. Think of the early days of libxl and the initial ARM support in
Linux. Both working, tested, but the ABI wasn't stable. Often asking
contributors to make the ABI stable to begin with could be setting the
bar too high for an initial contribution.

Something like:

"best effort" < "working (unstable ABI)" < "working (stable ABI)" < "supported"


> Having said all of this, I don't wish to deter feature submitters from
> striving for "Supported".  Attaining "Supported" should be a accolade
> that a feature is good from a technical point of view, with no important
> missing parts.  I will certainly continue to strive for my features to
> fit into this category, and I hope that the rest of the community will
> hold my code to the same high standard that I hold others to.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel