Re: [Xen-API] [MirageOS-devel] using Irmin to distribute the xapi database

[Thomas Gazagnaire <thomas@xxxxxxxxxxxxxx>]

Hi,

That's really great!

> If anyoneâs interested Iâve written up a basic design doc here:
> 
> https://github.com/xapi-project/xen-api/blob/master/doc/futures/distributed-database/README.md

I haven't looked at the code yet, but the design document makes a lot of sense: 
separating the strongly consistent lock server and the "branch"-consistent data 
store is the right way to go. Few minor remarks for now on, I'll surely have 
more later.

- If you design a system to sync a cluster of Irmin instances, that would be 
very nice to make it quite generic so it can be used by other Irmin users. 
Using a spanning tree makes sense there, but you want the event flow to go in 
both directions -- from and to the roors -- or you might want to use multiple 
spawning trees.

- I'm not sure about your use of views as branches. A quick reminder: views are 
like staging area in Git: they are hold in memory (so they don't persist if the 
host crash), reads are done lazily and writes are done only when needed (if if 
you modify a key twice, only the last change will be written to disk when you 
commit). Views also hold a list of operations, which are checked for conflicts 
on commits (and used to replay/rebase the view if needed). The most important 
feature of views is that they keep track of reads: ie. you can have a conflict 
if a view read a key which has been modified concurrently. 

On the opposite, branches are "normal" database state (the main one is a branch 
names "HEAD"). Every branch should have a different name and branches are 
persisted to disk. As for the main branch, you can see their full history of 
changes. Note: branches do not keep track of reads, so you can only have 
write/write (or write/remove or add/remove) conflicts. This can be a problem if 
you want to encode transactions (where you need to remember reads).

So having recursive branches is fine (as long as you have different branch 
names). Having recursive views should be fine as well (need to check the API 
again) but if that's not the case, that's easy enough to add. I'm not totally 
sure what is the semantics of a recursive view though.

Thomas 

> 
> Cheers,
> Dave
> 
> [1] https://github.com/xapi-project/xapi
> [2] https://github.com/mirage/irmin
> 
> 
> _______________________________________________
> MirageOS-devel mailing list
> MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
> http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api