Re: [Xen-API] [MirageOS-devel] xen dev summit talk slides: Branch Consistency for Xenstore

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

Thanks Felipe! (Huici and Manco :-)

Terrible wifi at the dev summit this year, so it was difficult to share more
information in real time.

best,
Anil

On 20 Aug 2014, at 04:05, Felipe Huici <Felipe.Huici@xxxxxxxxx> wrote:

> Hi Anil, all,
> 
> As a clarification, the talk was given by my colleague Filipe Manco, in
> cc, not me. I’ve temporarily placed the slides at:
> 
> http://www0.cs.ucl.ac.uk/staff/f.huici/presentations/clickos-xendevsummit14
> .pdf
> 
> They should appear on the Xen summit briefly, and more permanently on our
> website at:
> 
> https://cnp.neclab.eu
> 
> Thanks,
> 
> — Felipe
> 
> ============================================================
> Dr. Felipe Huici
> Senior Researcher, Networked Systems and Data
> Analytics Group
> NEC Laboratories Europe, Network Research Division
> Kurfuerstenanlage 36, D-69115 Heidelberg
> Tel.     +49
> (0)6221 4342-241
> Fax:     +49
> (0)6221 4342-155
> 
> e-mail: 
> felipe.huici@xxxxxxxxx
> ============================================================
> NEC Europe Limited Registered Office: NEC House, 1
> Victoria Road, London W3 6BL Registered in England 2832014
> 
> 
> 
> 
> On 8/19/14, 3:47 PM, "Anil Madhavapeddy" <anil@xxxxxxxxxx> wrote:
> 
>> On 19 Aug 2014, at 08:44, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>> 
>>> On 17 Aug 2014, at 11:01, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
>>> 
>>>> I'm giving a talk at the Xen Developer Summit at Linuxcon tomorrow on
>>>> the work that we've (primarily Dave Scott and Thomas Gazagnaire) on
>>>> improving the Xenstore via Irmin features.
>>>> 
>>>> Slides here: http://decks.openmirage.org/xendevsummit14#/
>>>> Repo: https://github.com/mirage/mirage-decks as usual
>>>> 
>>> 
>>> [+xen-api-devel] Talk went well, quick notes here:
>>> 
>>> - git workflow very popular. Lots of people twigged onto the
>>> maintainability benefits of `git bisect` automation in particular.
>>> 
>>> - questions about why Xenstore transactions are still necessary in the
>>> modern world.  Can replace with consensus protocols instead?  Maybe time
>>> for an ABI bump to deprecate the ancient xenstore protocol.
>>> 
>>> - space usage is a concern -- building an RRD-style constant size
>>> library to maintain progressive history would be a big win.
>>> 
>>> - *excellent* talk from Felipe Huici (CCed) from NEC about building
>>> much denser VM workloads, and he observed that Xenstored/xenconsoled are
>>> a big bottleneck at ~10000 VMs.  Are your slides available Felipe?  Some
>>> sub notes:
>>> 
>>> - we could write a mirage xenconsoled to log to irmin and drain guest
>>> console rings much faster.
>>> - a distributed xenstored+irmin would allow significantly more
>>> scalability than attempting to build a serially fast version.
>>> - felipe has the beginning of a simple c++ xenstored that doesn't
>>> implement the full semantics, but is enough for MiniOS.
>>> - it may be useful to negotiate a xenstore v2 protocol and use that for
>>> new guests.  It could use a simple fixed-length binary protocol
>>> (protobuf-style?), and eliminate the need for transactions perhaps.
>> 
>> sent 'send' too quickly.  The most important point:
>> 
>> - xenstore need consistent indices.  Using domain names in the current
>> stack results in a linear search through all domids -- very slow with
>> 10000 VMs!  Irmin could maintain a branch that updates a 'name -> domid'
>> that is consistent within the transaction, and do constant time lookups
>> for the toolstack.
>> 
>> -anil
>> 
> 


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api