Re: [Xen-API] [PATCH v5 3/3] Significant changes to decision making; some new roles and minor changes

[Lars Kurth <lars.kurth@xxxxxxxxxx>]

On 30/11/2016 23:27, "Stefano Stabellini" <sstabellini@xxxxxxxxxx> wrote:

>On Wed, 23 Nov 2016, Lars Kurth wrote:
>>
>> -Formal Votes {#formal-votes}
>> -------------
>> -
>> -Sometimes it is necessary to conduct formal voting within the
>>community 
>> -(outside of elections). Formal votes are necessary when processes and
>> -procedures are introduced or changed, or as part of the [Project
>> -Governance](#project-governance). Who is eligible to vote, depends on
>>whether 
>> -the scope of a process or procedure is **local** to a sub-project or
>>team, or 
>> -whether it affects **all sub-projects** (or in other words, is
>>**global**). 
>> -Examples of local scope is the [Security
>>Policy](/security-policy.html) which
>> -applies to the [Hypervisor Project](/developers/teams/hypervisor.html)
>>only. 
>> -Examples of global scope are changes to this document or votes
>>outlined in the 
>> -Project Governance.
>> -
>> -  
>>-------------------------------------------------------------------------
>>----
>> -  **Scope**    **Who reviews?**       **Who votes?**
>> -  ------------ ----------------------
>>-----------------------------------------
>> -  **Local**    Members of developer   Maintainers of the project (or
>>projects),
>> -               mailing lists of the   which are affected by the
>>process,
>> -               affected projects.     procedure, etc. are allowed to
>>vote.
>> -                                      This includes maintainers from
>>incubation 
>> -                                      projects (if the scope affects
>>the 
>> -                                      project).
>> -
>> -  **Global**   Members of all         Maintainers of **all mature**
>>projects 
>> -               developer mailing      and the Xenproject.org community
>>manager 
>> -               lists of all           are allowed to vote.
>> -               sub-projects hosted on
>> -               Xenproject.org.
>> -  
>>-------------------------------------------------------------------------
>>----
>> -\
>> +Projects which have a project lead, should vote for a project lead in
>>an 
>> +anonymous vote amongst the project leadership.
>> +
>> +### Project Wide Decision Making {#project-decisions}
>> +
>> +Project wide decisions are made through **formal global votes** and
>>are 
>> +conducted in rare circumstances only, following the principle of
>>[local 
>> +decision making](#principles). Global votes are only needed, when all
>>sub-projects 
>> +hosted on Xenproject.org are affected. This is true, only for:
>> +
>> +-   Specific votes on creating, graduating, completing/archiving of
>> +sub-projects as outlined in [project governance](#project-governance).
>> +-   Changes to this document, where sub-projects cannot specialise. In
>> +particular the sections: [goals](#goals), [principles](#principles),
>>[project 
>> +wide decision making](#project-decisions) and [project
>> +governance](#project-governance) (and small parts of [Xen Project wide
>> +roles](#roles-global), [project team roles](#roles-local) and
>>[decision 
>> +making](#decisions) that are needed for project governance or **apply
>>to all 
>> +sub-projects** as stated in those sections).
>> +-   Changes to this document where sub-projects can specialise require
>>at least 
>> +one mature project other than the Hypervisor project to be impacted
>> +significantly by the change. The sections in question, are [project
>>team 
>> +roles](#roles-local) and [decision making](#decisions). These sections
>>define 
>> +the **gold standard** of how the original Hypervisor Project operates.
>>In other 
>> +cases, the Hypervisor project leadership team can agree changes to
>>these 
>> +sections, as they are essentially reference definitions. Other
>>sub-projects 
>> +have to be consulted, and have to be given time to adapt to changes.
>> +-   Changes to existing global namespace policies (e.g. [Mailing List
>> 
>>+Conventions](/help/mailing-list/100-misc/139-mailing-list-conventions.ht
>>ml)) 
>> +and creation of new project wide namespace policies.
>> +-   Changes to the boundary of what policies are project local and
>>global 
>> +decision: e.g. a decision to introduce a global Security Vulnerability
>>Response 
>> +Process that affects all sub-projects.
>> +
>> +Global votes are arranged by the community manager when needed (e.g.
>>for a 
>> +project review or a global process change). Who exactly has input on a
>>proposal 
>> +and can vote on it, depends on the type of change as outlined below:
>> +
>> +  
>>-------------------------------------------------------------------------
>>---------------- 
>> +  **Proposal**                  **Who reviews?**              **Who
>>votes?**
>> +  ----------------------------- -----------------------------
>>-----------------------------
>> +  Creating, graduating,         Members of developer mailing
>>Leadership teams of
>> +  completing/archiving of       lists of qualifying projects
>>**mature** sub-projects,
>> +  sub-projects                                                with the
>>exception of the 
>> +                                                              project
>>which is being 
>> +                                                              reviewed
>>(e.g. for an 
>> +               
>>archivation review, the
>> +               
>>leadership team of the
>> +                                                              project
>>under review, cannot
>> +                                                              vote).
>> +
>> +  Global Process Changes        Members of developer mailing
>>Leadership teams of
>> +                                lists of qualifying projects
>>**mature** sub-projects,
>> +                                                              within
>>the scope described
>> +                                                              above.
>> +  
>>-------------------------------------------------------------------------
>>---------------- 
>> +
>>  
>>  The community manager first arranges a public review, followed by a
>>timed 
>>  private vote. Public review and voting should be open for a minimum of
>>a week 
>>  each. For voting a traceable poll mechanism (e.g. voting form that
>>keeps 
>> -auditable and tamper proof records) must be used. Voting follows the
>> -conventions as laid out in "Principle: Consensus Decision Making".
>> -
>> -Project Governance  {#project-governance}
>> +auditable and tamper proof records) must be used.
>> +
>> +Voting is conducted **per project** in line with the following rules:
>> +
>> +-   Each qualifying project's vote is counted per project and then
>>aggregated 
>> +as outlined below.
>> +-   Project leadership team members vote for or against a proposal
>>(there is no 
>> +differentiation between **-1**/**-2** and **+1**/**+2**). A **0** vote
>>is not 
>> +counted as a valid vote.
>> +-   A **quorum of at least least 1/3 of positive votes** of each
>>project's 
>> +leadership team members is required. In other words: if a project's
>>leadership 
>> +team does not achieve the quorum, the entire sub-project's vote is not
>>counted. 
>> +This avoids situations where only a minority of leadership team
>>members vote, 
>> +which would skew the overall result. If it becomes clear, that a
>>sub-project is 
>> +not likely to meet the quorum, the voting deadline can be extended by
>>the 
>> +community manager.
>> +
>> +__Passed/Failed Resolutions:__
>> +
>> +-   If none of the qualifying projects achieve a quorum, the change
>>cannot 
>> +hold. In that case, we consider that there is not enough momentum
>>behind a 
>> +change.
>> +-   For each qualifying project with a quorum, the percentage of votes
>>in 
>> +favour and against is calculated (e.g. if 5 people voted in favour, 2
>>against 
>> +and 1 abstains, the share is 5/7th and 2/7th respectively).
>> +-   Votes in favour are averaged as percentages across all projects
>>(say we 
>> +have per project figures of 50%, 80%, 70% in favour, then the total
>>vote in 
>> +favour is 66.67%).
>> +-   If the total vote achieves a 2/3rd majority in favour, the
>>proposal passes. 
>> +Otherwise it fails.
>> +
>
>This is basically the same voting mechanism described under "Leadership
>Team
>Decisions", counted per project, then averaged, isn't?

That is correct.

>It worries me that it could lead to very different results depending on
>the project leadership team sizes.
>
>For example, let's say that only 2 projects reach the quorum:
>project A, leadership team size 2, total positive votes 2, 100%
>project B, leadership team size 12, negative votes 8, positive votes 4,
>33%
>Total favor 66.5% -> pass (or very close to)

The issue that prompted this change was in effect created by the number of
committers in different mature projects (aka, the fact that XAPI has 12 -
14 - I have to verify the correct number, as some people in the XAPI
committer list don't work on XAPI any more). Where according to the
current scheme, projects with large leadership teams can in effect use
their larger voting block to get their opinion through.

One way of maybe addressing this, would be to be more specific about the
minimum size of a Leadership team (see "Projects without functional
Project Leadership Team"). I think a team needs to have at least 3 members
to be functional. Another way to add an extra check may be to add a
specific requirement to Graduation Review which checks that the Leadership
team is of an appropriate size for the size of the project (although we
may have to be specific on what an appropriate size is).

In reality, we don't have a problem with this today, as the leadership
teams for the two mature projects (XAPI and Hypervisor) are actually
large. We have
* 7 for the Hypervisor
* 12 for XAPI (although this is probably to big, but in reality
participation tends to be low)

The two projects which could qualify for maturity in the coming year are
Win PV drivers (3 leadership team members) and MirageOS (probably should
have a similar size to the Hypervisor Team).

Also, it is worthwhile pointing out, that Global Decisions should
practically hardly ever be needed. Only in the following situations
1) Creating, graduating, completing/archiving of sub-projects
2) Some changes to this document (goals, principles, project wide decision
making and project governance): if we apply the new rules, only this
change would need a global decision (as we added a principle and changed
local decision making). And this would be the first one, we had since
introducing the governance 5 years ago
3) Namespace issues: aka naming conventions for lists, ... - which
primarily would be bike-shed issues. But again we only used this once
4) Boundary issues: aka making local per-subproject policies and
conventions global 

>However I don't have a concrete suggestion on how to improve this. Given
>that any project could appoint any number of people in their leadership
>teams, I am not sure that accounting for the size of the teams would
>make things much better. On the other hand the number of people in the
>leadership team should represent the size of the project somewhat, so it
>could make sense to account for the votes proportionally.
>
>Any opinions?

The only other way I can think of is to weight a project's vote by some
level of activity (e.g. proportion of contributions averaged over 3
years). But that would become complicated.

Another way may be to add an extra bucket which contains all projects. In
the example above

project A, leadership team size 2, total positive votes 2, 100% (pass)
project B, leadership team size 12, negative votes 8, positive votes 4,
33% (fail)
ALL (which is like the popular vote): size 14, negative votes 8, positive
votes 6, 42% (fail)
Average 58% (or very close to) -> fail  ... which does change this example


Or some sort of rule, which requires that the popular and aggregated votes
have to be within a certain percentage of each other, otherwise the vote
does not count and has to be repeated

>For everything else, you have my +1.
>For this section, I'll think about it a bit more :-)

Please do

Best Regards
Lars

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
https://lists.xen.org/cgi-bin/mailman/listinfo/xen-api