Open Stack

On Mar 6, 2014, at 9:56 PM, Zhangleiqiang <zhangleiqiang at huawei.com> wrote:

>> get them working. For example, in a devstack VM the only way I can get the
>> iSCSI target to show the new size (after an lvextend) is to delete and recreate
>> the target, something jgriffiths said he doesn't want to support ;-).
> 
> I know a method can achieve it, but it maybe need the instance to pause first (during the step2 below), but without detaching/reattaching. The steps as follows:
> 
> 1. Extend the LV
> 2.Refresh the size info in tgtd:
>  a) tgtadm --op show --mode target # get the "tid" and "lun_id" properties of target related to the lv; the "size" property in output result is still the old size before lvextend
>  b) tgtadm --op delete --mode logicalunit --tid={tid} --lun={lun_id}  # delete lun mapping in tgtd
>  c) tgtadm --op new --mode logicalunit --tid={tid} --lun={lun_id} --backing-store=/dev/cinder-volumes/{lv-name} # re-add lun mapping

Sure, this is my current workaround, but it's what I thought we *didn't* want to have to do.

>  d) tgtadm --op show --mode target #now the "size" property in output result is the new size
> *PS*:  
> a) During the procedure, the corresponding device on the compute node won't disappear. But I am not sure the result if Instance has IO on this volume, so maybe the instance may be paused during this procedure.

Yeah, but pausing the instance isn't an online extend. As soon as the user can't interact with their instance, even briefly, it's an offline extend in my view.

> b) Maybe we can modify tgtadm, and make it support the operation which is just "refresh" the size of backing store.

Maybe. I'd be interested in any thoughts/patches you have to accomplish this. :)

> 
> 3. Rescan the lun info in compute node: iscsiadm -m node --targetname {target_name} -R

Yeah, right now as part of this work I'm adding two extensions to Nova. One to issue this rescan on the compute host and another to get the size of the block device so Cinder can poll until the device is actually the new size (not an ideal solution, but so far I don't have a better one).

> 
>> I also
>> haven't dived into any of those other limits you mentioned (nfs_used_ratio,
>> etc.).
> 
> Till now, we focused on the "volume" which is based on *block device*. Under this scenario, we must first try to "extend" the volume and notify the hypervisor, I think one of the preconditions is to make sure the extend operation will not affect the IO in Instance.
> 
> However, there is another scenario which maybe a little different. For *online-extend" virtual disks (qcow2, sparse, etc) whose backend storage is file system (ext3, nfs, glusterfs, etc), the current implementation of QEMU is as follows:
> 1. QEMU drain all IO
> 2. *QEMU* extend the virtual disk
> 3. QEMU resume IO
> 
> The difference is the *extend* work need be done by QEMU other than cinder driver. 
> 
>> Feel free to ping me on IRC (pdmars).
> 
> I don't know your time zone, we can continue the discussion on IRC, :)

Good point. :) I'm in the US central time zone.

Paul

> 
> ----------
> zhangleiqiang
> 
> Best Regards
> 
> 
>> -----Original Message-----
>> From: Paul Marshall [mailto:paul.marshall at RACKSPACE.COM]
>> Sent: Thursday, March 06, 2014 12:56 AM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Cc: Luohao (brian)
>> Subject: Re: [openstack-dev] [Cinder] Do you think we should introduce the
>> online-extend feature to cinder ?
>> 
>> Hey,
>> 
>> Sorry I missed this thread a couple of days ago. I am working on a first-pass of
>> this and hope to have something soon. So far I've mostly focused on getting
>> OpenVZ and the HP LH SAN driver working for online extend. I've had trouble
>> with libvirt+kvm+lvm so I'd love some help there if you have ideas about how to
>> get them working. For example, in a devstack VM the only way I can get the
>> iSCSI target to show the new size (after an lvextend) is to delete and recreate
>> the target, something jgriffiths said he doesn't want to support ;-). I also
>> haven't dived into any of those other limits you mentioned (nfs_used_ratio,
>> etc.). Feel free to ping me on IRC (pdmars).
>> 
>> Paul
>> 
>> 
>> On Mar 3, 2014, at 8:50 PM, Zhangleiqiang <zhangleiqiang at huawei.com>
>> wrote:
>> 
>>> @john.griffith. Thanks for your information.
>>> 
>>> I have read the BP you mentioned ([1]) and have some rough thoughts about
>> it.
>>> 
>>> As far as I know, the corresponding online-extend command for libvirt is
>> "blockresize", and for Qemu, the implement differs among disk formats.
>>> 
>>> For the regular qcow2/raw disk file, qemu will take charge of the drain_all_io
>> and truncate_disk actions, but for raw block device, qemu will only check if the
>> *Actual* size of the device is larger than current size.
>>> 
>>> I think the former need more consideration, because the extend work is done
>> by libvirt, Nova may need to do this first and then notify Cinder. But if we take
>> allocation limit of different cinder backend drivers (such as quota,
>> nfs_used_ratio, nfs_oversub_ratio, etc) into account, the workflow will be
>> more complicated.
>>> 
>>> This scenario is not included by the Item 3 of BP ([1]), as it cannot be simply
>> "just work" or notified by the compute node/libvirt after the volume is
>> extended.
>>> 
>>> This regular qcow2/raw disk files are normally stored in file system based
>> storage, maybe the Manila project is more appropriate for this scenario?
>>> 
>>> 
>>> Thanks.
>>> 
>>> 
>>> [1]:
>> https://blueprints.launchpad.net/cinder/+spec/inuse-extend-volume-extension
>>> 
>>> ----------
>>> zhangleiqiang
>>> 
>>> Best Regards
>>> 
>>> From: John Griffith [mailto:john.griffith at solidfire.com]
>>> Sent: Tuesday, March 04, 2014 1:05 AM
>>> To: OpenStack Development Mailing List (not for usage questions)
>>> Cc: Luohao (brian)
>>> Subject: Re: [openstack-dev] [Cinder] Do you think we should introduce the
>> online-extend feature to cinder ?
>>> 
>>> 
>>> 
>>> 
>>> On Mon, Mar 3, 2014 at 2:01 AM, Zhangleiqiang
>> <zhangleiqiang at huawei.com> wrote:
>>> Hi, stackers:
>>> 
>>>        Libvirt/qemu have supported online-extend for multiple disk
>> formats, including qcow2, sparse, etc. But Cinder only support offline-extend
>> volumes currently.
>>> 
>>>    Offline-extend volume will force the instance to be shutoff or the volume
>> to be detached. I think it will be useful if we introduce the online-extend feature
>> to cinder, especially for the file system based driver, e.g. nfs, glusterfs, etc.
>>> 
>>>    Is there any other suggestions?
>>> 
>>>    Thanks.
>>> 
>>> 
>>> ----------
>>> zhangleiqiang
>>> 
>>> Best Regards
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> Hi Zhangleiqiang,
>>> 
>>> So yes, there's a rough BP for this here: [1], and some of the folks from the
>> Trove team (pdmars on IRC) have actually started to dive into this.  Last I
>> checked with him there were some sticking points on the Nova side but we
>> should synch up with Paul, it's been a couple weeks since I've last caught up
>> with him.
>>> 
>>> Thanks,
>>> John
>>> [1]:
>> https://blueprints.launchpad.net/cinder/+spec/inuse-extend-volume-extension
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev