Re: [Xen-API] [xs-devel] lvmthin

[Dave Scott <Dave.Scott@xxxxxxxxxx>]

> On 12 Jan 2015, at 09:29, Pasi KÃrkkÃinen <pasik@xxxxxx> wrote:
> 
> On Fri, Jan 09, 2015 at 03:59:26PM -0700, Tobias Kreidl wrote:
>>   So sorry, I meant of course to write that the utility is lvmthin (not
>>   thinlvm).
>> 
> 
> Hello,
> 
> I *think* the lvmthin stuff is currently designed for single-host systems,
> where the storage is used by a single dedicated host only. That's the usual 
> model for LVM aswell.

Thatâs right.

> 
> If multiple hosts access the same LVM volumes one needs to make sure the LVM 
> metadata changes are properly synchronized across all hosts,
> so all the hosts sharing the same physical LUNs/PVs/VGs/LVs have exactly the 
> same idea of the LVM settings, and I believe XAPI does this for normal 
> (thick) LVM volumes.
> 
> But it gets much more difficult with thin volumes, because the "metadata" can 
> change on *every* write IO (when one needs to allocate more blocks from the 
> thin pool?).
> 
> Note I haven't looked at the internals of LVM thinprovisioning, but I assume 
> it'll be more difficult to get working in shared-LUN/multi-host environments 
> compared to normal thick-LVM.
> 
> I guess there would have to be some "free-blocks-per-host-for-thin-volumes", 
> so each host would allocate new blocks from its dedicated pool, without the 
> risk of corrupting the LVM PVs or the volumes.. and avoiding the need of 
> syncing metadata on every write IO?

LVMâs thin provisioning uses the device mapper âdm-thinâ target, which works as 
you describe. I think for local, non-shared LVM it looks really good. Iâve got 
a very experimental storage plugin which can use it (ezlvm[1]). Also I think 
Fedora 21 can boot from thin-provisioned LVM which would let you do things like 
snapshot (and revert) the whole dom0 filesystem.

Iâve thought a little about how we could use it in future for shared LVM. 
Potentially we could make the âthin poolâ work for per-host allocation by a 
ballooning-like technique. If every host saw the full set of free blocks, but 
most of the blocks were masked off by a fake disk (like the balloon driverâs 
balloon) then each host would be able to allocate locally from the same block 
address space. This would make it easier to move volumes between hosts. I think 
each host would need its own private copy of the LVM metadata, which would 
function like the journal in this âthinlvhdâ design. When I tried to make this 
work a while ago I hit a problem when I tried to modify the âballoon diskâ: 
when I reloaded the thin pool metadata and resumed the device mapper device it 
had clearly cached some of the data structure in memory because it immediately 
corrupted itself. This is probably fixable but requires more experimentation :)

So for the shared case I think we should work on âthinlvhdâ (i.e. thin 
provisioning via tapdisk) first but play with dm-thin in the background :)

Cheers,
Dave

[1] https://github.com/xapi-project/ezlvm


> 
> 
> Just random thoughts :)
> 
> -- Pasi
> 
>>   On 1/9/2015 3:54 PM, Tobias Kreidl wrote:
>> 
>>     As of around CentOS 6.5 (and for sure in RHEL 6.4) that there is a
>>     thinlvm lvmthin utility that seems to take care of supporting
>>     thinly-provisioned LVM volumes. There is an associated snapshot design
>>     based on LVM thin provisioning that even supports the ability to do
>>     snapshots of snapshots, etc. down the chain.
>> 
>>     Unfortunately, from a cursory look, it doesn't look like a back port to
>>     CentOS 5 would be that easy or even possible, but that it is integrated
>>     into both CentOS 6 and CentOS 7 gives some hope for a possibly
>>     standardized support of it in a future XenServer release, doesn't it?
>>     The other big question would be how readily something like this could be
>>     integrated into XenServer.
>> 
>>     -=Tobias
> 

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api