[openstack-dev] Cinder: Whats the way to do cleanup during service shutdown / restart ?

Deepak Shetty dpkshetty at gmail.com
Mon Apr 7 05:13:02 UTC 2014


To add:
    I was looking at Nova code and it seems there is a framework for
cleanup using the terminate calls.. IIUC this works as libvirt calls
terminate on Nova instance when the VM is shutting down/destroying, hence
terminate seems to be a good place to do cleanup on Nova side.. something
similar is missing on Cinder side and __del__ way of cleanup isn't working
as I posted above.


On Mon, Apr 7, 2014 at 10:24 AM, Deepak Shetty <dpkshetty at gmail.com> wrote:

> Duncan,
>     Thanks for your response. Tho' i agree to what you said.. I am still
> trying to understand why i see what i see .. i.e. why the base class
> variables (_mount_shared) shows up empty in __del__
> I am assuming here that the obj is not completely gone/deleted, so its
> vars must still be in scope and valid.. but debug prints suggests the
> otherwise :(
>
>
> On Sun, Apr 6, 2014 at 12:07 PM, Duncan Thomas <duncan.thomas at gmail.com>wrote:
>
>> I'm not yet sure of the right way to do cleanup on shutdown, but any
>> driver should do as much checking as possible on startup - the service
>> might not have gone down cleanly (kill -9, SEGFAULT, etc), or
>> something might have gone wrong during clean shutdown. The driver
>> coming up should therefore not make any assumptions it doesn't
>> absolutely have to, but rather should check and attempt cleanup
>> itself, on startup.
>>
>> On 3 April 2014 15:14, Deepak Shetty <dpkshetty at gmail.com> wrote:
>> >
>> > Hi,
>> >     I am looking to umount the glsuterfs shares that are mounted as
>> part of
>> > gluster driver, when c-vol is being restarted or Ctrl-C'ed (as in
>> devstack
>> > env) or when c-vol service is being shutdown.
>> >
>> > I tried to use __del__ in GlusterfsDriver(nfs.RemoteFsDriver) and it
>> didn't
>> > work
>> >
>> > def __del__(self):
>> >         LOG.info(_("DPKS: Inside __del__ Hurray!, shares=%s")%
>> > self._mounted_shares)
>> >         for share in self._mounted_shares:
>> >             mount_path = self._get_mount_point_for_share(share)
>> >             command = ['umount', mount_path]
>> >             self._do_umount(command, True, share)
>> >
>> > self._mounted_shares is defined in the base class (RemoteFsDriver)
>> >
>> > ^C2014-04-03 13:29:55.547 INFO cinder.openstack.common.service [-]
>> Caught
>> > SIGINT, stopping children
>> > 2014-04-03 13:29:55.548 INFO cinder.openstack.common.service [-] Caught
>> > SIGTERM, exiting
>> > 2014-04-03 13:29:55.550 INFO cinder.openstack.common.service [-] Caught
>> > SIGTERM, exiting
>> > 2014-04-03 13:29:55.560 INFO cinder.openstack.common.service [-]
>> Waiting on
>> > 2 children to exit
>> > 2014-04-03 13:29:55.561 INFO cinder.openstack.common.service [-] Child
>> 30185
>> > exited with status 1
>> > 2014-04-03 13:29:55.562 INFO cinder.volume.drivers.glusterfs [-] DPKS:
>> > Inside __del__ Hurray!, shares=[]
>> > 2014-04-03 13:29:55.563 INFO cinder.openstack.common.service [-] Child
>> 30186
>> > exited with status 1
>> > Exception TypeError: "'NoneType' object is not callable" in <bound
>> method
>> > GlusterfsDriver.__del__ of
>> <cinder.volume.drivers.glusterfs.GlusterfsDriver
>> > object at 0x2777ed0>> ignored
>> > [stack at devstack-vm tempest]$
>> >
>> > So the _mounted_shares is empty ([]) which isn't true since I have 2
>> > glsuterfs shares mounted and when i print _mounted_shares in other
>> parts of
>> > code, it does show me the right thing.. as below...
>> >
>> > From volume/drivers/glusterfs.py @ line 1062:
>> >     LOG.debug(_('Available shares: %s') % self._mounted_shares)
>> >
>> > which dumps the debugprint  as below...
>> >
>> > 2014-04-03 13:29:45.414 DEBUG cinder.volume.drivers.glusterfs
>> > [req-2cf69316-cc42-403a-96f1-90e8e77375aa None None] Available shares:
>> > [u'devstack-vm.localdomain:/gvol1', u'devstack-vm.localdomain:/gvol1']
>> from
>> > (pid=30185) _ensure_shares_mounted
>> > /opt/stack/cinder/cinder/volume/drivers/glusterfs.py:1061
>> >
>> > This brings in few Qs ( I am usign devstack env) ...
>> >
>> > 1) Is __del__ the right way to do cleanup for a cinder driver ? I have 2
>> > gluster backends setup, hence 2 cinder-volume instances, but i see
>> __del__
>> > being called once only (as per above debug prints)
>> > 2) I tried atexit and registering a function to do the cleanup.
>> Ctrl-C'ing
>> > c-vol (from screen ) gives the same issue.. shares is empty ([]), but
>> this
>> > time i see that my atexit handler called twice (once for each backend)
>> > 3) In general, whats the right way to do cleanup inside cinder volume
>> driver
>> > when a service is going down or being restarted ?
>> > 4) The solution should work in both devstack (ctrl-c to shutdown c-vol
>> > service) and production (where we do service restart c-vol)
>> >
>> > Would appreciate a response
>> >
>> > thanx,
>> > deepak
>> >
>> >
>> > _______________________________________________
>> > OpenStack-dev mailing list
>> > OpenStack-dev at lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>>
>>
>> --
>> Duncan Thomas
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140407/d132c1e4/attachment.html>


More information about the OpenStack-dev mailing list