Re: [Xen-devel] [RFC PATCH 0/4] Add missing default labels to switch statements

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 27 Feb 2019, Lars Kurth wrote:
> > On 27 Feb 2019, at 10:16, Jan Beulich <jbeulich@xxxxxxxx> wrote:
> > 
> >>>> On 27.02.19 at 10:23, <lars.kurth.xen@xxxxxxxxx> wrote:
> >> 
> >> I recall that I read in an earlier thread that Julien and Stefano have 
> >> access to the document, which would leave Jan and a few members of Citrix 
> >> staff. Can those committers who need access raise their hands? I can then 
> >> go 
> >> ahead and order these.
> > 
> > Well, you've effectively raised my hand already. To be honest I'm not
> > sure I want it raised: I fear to break in tears when I would get to read
> > that book. In any event, I'd say ...
> 
> It's a reference document to look up stuff. Not something you would 
> necessarily read upfront.
> 
> >> Having followed this thread (and the other MISRA related one from 
> >> Stefano), 
> >> it seems to me that potentially each of these discussions is quite 
> >> divisive 
> >> and take up a lot of discussion and emotional energy. With 143 rules and 
> >> 16 
> >> "directives" (more like guidance) and some of the rules being mandatory 
> >> (73%) 
> >> and some advisory (27%), but the possibility to justify deviating from the 
> >> rule, maybe we are approaching this wrongly. 
> >> 
> >> I have some thoughts about the approach and will follow up on this thread 
> >> later today or tomorrow when I had some more time to clarify my thoughts.
> > 
> > ... don't order anything before we aren't clear whether we really want
> > to do this (or even any part thereof) to the code base.
> 
> Alright: firstly I need to explain that I asked EPAM to start looking a half 
> dozen or so "interesting" Misra compliance issues and post RFC patches. The 
> idea behind this was to gather data about how as a community we would handle 
> these  kind of issues. There was a discussion about Misra (or safety related 
> coding standards in general) at last years developer summit, which went 
> nowhere 
> due to lack of data. 
> 
> It is clear to me that as a community we have to deal with Misra C compliance 
> and other efforts to make Xen more easily safety certifiable seriously and 
> can't just wish it to go away. I think it is fair to say that the project is 
> facing increased competition from KVM and containers, while at the same time 
> Xen has unique advantages that lead vendors to go down the embedded/safety 
> route. If, as a community we just dismiss these efforts, we risk a fork or 
> those vendors going elsewhere. Neither would be good for the community.
> 
> Having seen the two discussions so far, it appears that even when we agree 
> that there is an issue, we seem to have real issues agreeing on workable 
> solutions. I also already had complaints that these threads generate to much 
> discussion (aka "noise").
> 
> What I don't know, is whether the two issues posted (this one and 
> https://markmail.org/message/ni3yziazuwb2aolx) are representative for the 
> kind 
> of issue we need to fix to achieve on Misra compliance, or whether they are 
> difficult outliers.
> 
> @Oleksandr: maybe you have some insights 
> 
> So the question is how we should approach this:
> 
> 1: One is to follow what we do now - post patches per issue and work through 
>    them. This only really scales if the majority of patches are in essence
>    uncontroversial.
> 
> 2: A slightly different approach would be for EPAM to post a few more 
> examples 
>    of the type of issues that we would have to deal with if we want to be 
> MISRA
>    compliant. But that we exercise restraint in the discussion knowing that 
> these
>    are examples to inform a discussion at https://wiki.xenproject.org/wiki/
>    Developer_Meeting/March2019_-_Safety_Certification and possible follow-up.
> 
>    What I was after when I asked EPAM to post Misra related patches was to
>    get a sense of the impact and a sense of how easily resolvable issues are.
>    But I wouldn't expect a full resolution at this stage, if there
>    is controversy. 
> 
>    So maybe we can handle these in a different way. From my PoV, it would be 
> good 
>    enough if key reviewers communicated per example whether
>    - They accept that fixing the issue would be beneficial
>    - What concerns they have
>    - And how much they would fight for or against such a patch
>      (using the -2 ... +2 scale as outlined in "EXPRESSING AGREEMENT AND 
>      DISAGREEMENT" in https://xenproject.org/developers/governance/#decisions
>    
>    Clearly there can be some discussion, but we don't really need to "fight
>    to the end" over these. 
> 
> 3: Or we could change approach completely and go for a more high-level
>    design and/or analysis based approach before we do anything else. I will 
> expand 
>    further down.
> 
> My personal preference would be to use 2 for a few patches, followed by 
> 3 as it gives us a different perspective.
> 
> Let me outline my thinking on 3:
> 
> There are a few things about Misra that we do not yet fully understand on a
> number of different dimensions:
> a) Issues are either mandatory or advisory. The scale changes depending on 
>    the required level of safety (expressed in ASIL A-D).
> b) There will likely be clusters of Misra rules we likely violate frequently
>    and others we are hardly or not affected by   
> 
> We should be able to pull an overview together using the QA Verify tool
> maybe initially filtering out rule violations which are advisory in the 
> context 
> of the goal of achieving ASIL A/B.
> 
> We could have a discussion about these in some sort of design document which
> covers the rule violations and proposes ways on how these would be addressed
> maybe with some examples. This would be less labour intensive than preparing
> actual patches and would keep things in one place. We can even break this into
> small chunks (maybe sorted by the frequency it affects the code)
> 
> IMPORTANT:
> c) There is also a provision to justify that certain Misra rules should not 
>    apply for a specific code base. This is called a DEVIATION. It appears to 
> be 
>    that frequently 20-30% of rule breakages are justified away.
> 
>    However, the DEVIATIONS are typically approved by a certification body
>    which decides that some justifications have merit, while others don't. The
>    problem we have is that we have no idea what kind DEVIATIONS and 
>    justifications we can get away with.
> 
> Again: I hope that some of this will become clearer at the safety meeting in 
> March, where we will have safety certification experts and community members
> working together. Maybe we can even convince one of those experts to act as
> an advisor/reviewer for the project.
> 
> To do ANY of this, does however require access to the Misra rules. 
> 
> Does this make sense? 
> 
> Would you be able to follow my suggested approach?

I agree with you on the suggested approaches. I would like to add that
although we are making progress with the current approach (1), it has a
very low yield, leading to very high costs to reach a positive
conclusion. It is not sustainable, neither as a community nor as
contributors/maintainers.

My second thought is that we need to have a way to deal with "creative
compliance" suggestions, such as [1]. Your idea of having a
certification expert on xen-devel could be a way to solve this.

[1] https://marc.info/?l=xen-devel&m=155111648202790 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel