Re: [Xen-API] [MirageOS-devel] xen dev summit talk slides: Branch Consistency for Xenstore

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 19 Aug 2014, at 08:44, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:

> On 17 Aug 2014, at 11:01, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:
> 
>> I'm giving a talk at the Xen Developer Summit at Linuxcon tomorrow on the 
>> work that we've (primarily Dave Scott and Thomas Gazagnaire) on improving 
>> the Xenstore via Irmin features.
>> 
>> Slides here: http://decks.openmirage.org/xendevsummit14#/
>> Repo: https://github.com/mirage/mirage-decks as usual
>> 
> 
> [+xen-api-devel] Talk went well, quick notes here:
> 
> - git workflow very popular. Lots of people twigged onto the maintainability 
> benefits of `git bisect` automation in particular.
> 
> - questions about why Xenstore transactions are still necessary in the modern 
> world.  Can replace with consensus protocols instead?  Maybe time for an ABI 
> bump to deprecate the ancient xenstore protocol.
> 
> - space usage is a concern -- building an RRD-style constant size library to 
> maintain progressive history would be a big win.
> 
> - *excellent* talk from Felipe Huici (CCed) from NEC about building much 
> denser VM workloads, and he observed that Xenstored/xenconsoled are a big 
> bottleneck at ~10000 VMs.  Are your slides available Felipe?  Some sub notes:
> 
> - we could write a mirage xenconsoled to log to irmin and drain guest console 
> rings much faster.
> - a distributed xenstored+irmin would allow significantly more scalability 
> than attempting to build a serially fast version.
> - felipe has the beginning of a simple c++ xenstored that doesn't implement 
> the full semantics, but is enough for MiniOS.
> - it may be useful to negotiate a xenstore v2 protocol and use that for new 
> guests.  It could use a simple fixed-length binary protocol 
> (protobuf-style?), and eliminate the need for transactions perhaps.

sent 'send' too quickly.  The most important point:

- xenstore need consistent indices.  Using domain names in the current stack 
results in a linear search through all domids -- very slow with 10000 VMs!  
Irmin could maintain a branch that updates a 'name -> domid' that is consistent 
within the transaction, and do constant time lookups for the toolstack.

-anil


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api