[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-API] VDI import/export with deltas

Hi Mate,

On 29 Aug 2014, at 09:58, Mate Lakat <mate.lakat@xxxxxxxxxx> wrote:

> Hi,
> I'm happy to see a prototype for import/export. It seems that this discussion 
> is generating a lot of traffic - do we plan to have some sort of design 
> session about it? I believe the storage team and the stackers would love to 
> be involved in these sessions.
> We also had a discussion which might be releated:
> https://lists.xenserver.org/sympa/arc/xs-devel/2013-08/thrd2.html#00059

We did, thanks for the pointer.

I’ve also done a survey of import/export code in xapi:


The prototype is basically ‘what we already have’: the code was all written for 
various other purposes (e.g. storage motion and CloudStack integration) and all 
I did was to fill in a few missing gaps and add CLI commands on top of existing 

> I think the idea of a filesystem interface is very good, however given that 
> the filesystem must provide a consistent view and transactions, as other 
> processes might manipulate files (GC), I think for the moment, let's try to 
> do a bit narrower interface.

I’m definitely in favour of a narrow interface :-)
At the moment our interface is far too wide and exposes far too much (not 
interesting but still important) internal details.

> I also wanted to raise the content ID question - would be nice to generate a 
> content id during the export, and mark it as dirty whenever it has changed, 
> so that the client would know that she should not expect a consistent output.

That would be good. One of the things the current code can’t do — but that I’m 
sure we discussed before — is to support resumable downloads.


> Cheers,
> Mate
> -----Original Message-----
> From: Dave Scott 
> Sent: Thursday, August 28, 2014 5:10 PM
> To: Bob Ball
> Cc: xen-api@xxxxxxxxxxxxxxxxxxxx; Mate Lakat; John Garbutt; Ant Messerli
> Subject: Re: VDI import/export with deltas
> On 28 Aug 2014, at 15:50, Bob Ball <bob.ball@xxxxxxxxxx> wrote:
>>> http://wiki.xensource.com/wiki/Disk_import/export_APIs
>> This is great, but I think we're missing the ability to pull a VHD in or 
>> push a VHD out.
>> The OpenStack case needs to call a vm-import giving a URL to pull a VHD 
>> from, and a vm-export to push a VHD out to.
>> In particular I think we need to set specific headers as well, as 
>> authentication, so I'd like to propose the following as a hopefully simple 
>> extension:
>> xe vdi-export uuid=$VDI format=vhd dest= 
>> headers:arbitrary_string=$AUTH_TOKEN
>> xe vdi-import uuid=$RESTORE format=vhd 
>> source= 
>> headers:arbitrary_string=$AUTH_TOKEN
> This looks sensible to me; plus I'd like to use URLs more in the storage 
> interface in future.
>> OpenStack also has a case for using bittorrent to download the VDI from 
>> peers rather than the central server, but I don't see how to accomplish that 
>> with the current proposal without requiring double the disk space as we'd 
>> need to download it then push it into XS.
> So at the moment, do you receive the .vhd data by bittorrent in-place?
> When I last spoke to John about this we discussed some kind of import/export 
> plugin extension. I guess to do the most efficient thing possible it would 
> have to have access to (at least a subset of) the physical storage, so it can 
> 'drop in' a vhd (or qcow or vmdk). Ideally the plugin wouldn't see anything 
> else (so it couldn't get confused or make a mistake) and the modified files 
> would be health-checked when the operation is finished.
>> Thoughts?
> Maybe we could make a safe import/export virtual filesystem with FUSE? 
> Perhaps you could even SMB export a virtual folder for XenCenter users on 
> windows to drag 'n drop files into.
> Cheers,
> Dave
>> Thanks,
>> Bob
>>> There's still time before 2.0 to change how these work, so your 
>>> comments and criticism would be appreciated.
>>> Thanks!
>>> Dave
>>> [1] https://github.com/xapi-project/xapi-project/issues/1
>>> [2] https://github.com/xapi-project/xen-api/issues/1778
>>> [3] https://github.com/xapi-project/vhd-tool/pull/14
>>> _______________________________________________
>>> Xen-api mailing list
>>> Xen-api@xxxxxxxxxxxxx
>>> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Xen-api mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.