|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-API] [MirageOS-devel] xen dev summit talk slides: Branch Consistency for Xenstore
>>> I'm giving a talk at the Xen Developer Summit at Linuxcon tomorrow on the >>> work that we've (primarily Dave Scott and Thomas Gazagnaire) on improving >>> the Xenstore via Irmin features. >>> >>> Slides here: http://decks.openmirage.org/xendevsummit14#/ >>> Repo: https://github.com/mirage/mirage-decks as usual >>> >> >> [+xen-api-devel] Talk went well, quick notes here: >> >> - git workflow very popular. Lots of people twigged onto the maintainability >> benefits of `git bisect` automation in particular. >> >> - questions about why Xenstore transactions are still necessary in the >> modern world. Can replace with consensus protocols instead? Maybe time for >> an ABI bump to deprecate the ancient xenstore protocol. >> >> - space usage is a concern -- building an RRD-style constant size library to >> maintain progressive history would be a big win. A note on that: during his internship, Benjamin Farinier wrote a very simple backend for Irmin which maps Irmin objects to the OCaml heap directly[1], which means that 1/ no SHA1 needs to be computed (so it's fast) and 2/ we picky-bag the OCaml GC to collect Irmin objects. I think this is a good base for an in-memory store, we just need to understand how to expose it efficiently to the Git backend (ie. you don't want to recompute all the SHA1 on every export). But yes, having an RRD-style history is the way to go. [1] https://github.com/BFarinier/irmin/blob/master/examples/objAO.ml#L7 >> - *excellent* talk from Felipe Huici (CCed) from NEC about building much >> denser VM workloads, and he observed that Xenstored/xenconsoled are a big >> bottleneck at ~10000 VMs. Are your slides available Felipe? Some sub notes: >> >> - we could write a mirage xenconsoled to log to irmin and drain guest >> console rings much faster. >> - a distributed xenstored+irmin would allow significantly more scalability >> than attempting to build a serially fast version. >> - felipe has the beginning of a simple c++ xenstored that doesn't implement >> the full semantics, but is enough for MiniOS. >> - it may be useful to negotiate a xenstore v2 protocol and use that for new >> guests. It could use a simple fixed-length binary protocol >> (protobuf-style?), and eliminate the need for transactions perhaps. > > sent 'send' too quickly. The most important point: > > - xenstore need consistent indices. Using domain names in the current stack > results in a linear search through all domids -- very slow with 10000 VMs! > Irmin could maintain a branch that updates a 'name -> domid' that is > consistent within the transaction, and do constant time lookups for the > toolstack. Irmin use persistent map to store sub-paths, so we already have logarithmic lookup (initially it was stored in a list, but my initial benchmarks pointed that as a bottleneck very quickly). Using maps is very efficient in practice. Thomas > > -anil > > > _______________________________________________ > MirageOS-devel mailing list > MirageOS-devel@xxxxxxxxxxxxxxxxxxxx > http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel _______________________________________________ Xen-api mailing list Xen-api@xxxxxxxxxxxxx http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |