[Openstack] Ceph bootable cinder volume not working in Havana
Alvin Starr
alvin at netvel.net
Thu Dec 12 14:08:55 UTC 2013
I found my problem.
/usr/lib64/qemu/librbd.so.1 was missing.
mkdir /usr/lib64/qemu/
ln -s /usr/lib64/librbd.so.1 /usr/lib64/qemu/librbd.so.1
On 12/11/2013 11:33 PM, Haomai Wang wrote:
> On Thu, Dec 12, 2013 at 2:29 AM, Alvin Starr <alvin at netvel.net> wrote:
>> I have noticed a similar problem.
>>
>> I have 2 different pools:
>> images and volumes and both use the same ceph.user keyring.
>>
>> I can create a volume from an image but if I try to cause the image to copy
>> as part of an instance start up things. fail.
>>
>>
>> I wonder if the problem is using 2 different pools?
>>
> It doesn't matter if the user own the same enough privileges.
>
>> On 12/11/2013 09:58 AM, Edward Lezondra wrote:
>>> This is my cinder.conf
>>>
>>> [DEFAULT]
>>>
>>>
>>> api_paste_config=/etc/cinder/api-paste.ini
>>> state_path = /var/lib/cinder
>>> glance_host=localhost
>>> glance_api_version=2
>>>
>>> rootwrap_config=/etc/cinder/rootwrap.conf
>>> auth_strategy=keystone
>>>
>>> lock_path=/var/lock/cinder
>>>
>>> qpid_hostname=localhost
>>> qpid_port=5672
>>> qpid_heartbeat=60
>>> qpid_tcp_nodelay=true
>>>
>>> iscsi_ip_address=10.198.0.191
>>> iscsi_helper=tgtadm
>>>
>>> volumes_dir = /var/lib/cinder/volumes
>>> volume_group=cinder-volumes
>>>
>>> rbd_pool=nova
>>> rbd_user=nova
>>> rbd_ceph_conf=/etc/ceph/ceph.conf
>>> rbd_secret_uuid=ae396956-cf03-483f-b356-2f447dc7a637
>>> volume_driver=cinder.volume.drivers.rbd.RBDDriver
>>>
>>> sql_connection = mysql://cinder:notcinder@173.194.106.60/cinder
>>> my_ip = 10.198.0.191
>>> rpc_backend = cinder.openstack.common.rpc.impl_qpid
>>> qpid_hostname = 10.198.0.191
>>> notification_driver=cinder.openstack.common.notifier.rpc_notifier
>>>
>>> Edward Lezondra
>>> IT
>>>
>>> IMC Financial Markets | 233 South Wacker Drive # 4300 | Chicago, IL 60606
>>> | http://www.imc-chicago.com
>>> Phone: +13122755425 | E-Mail: Edward.Lezondra at imc-chicago.com
>>>
>>> -----Original Message-----
>>> From: Haomai Wang [mailto:haomaiwang at gmail.com]
>>> Sent: Monday, December 09, 2013 10:03 PM
>>> To: Edward Lezondra
>>> Cc: Openstack at lists.openstack.org
>>> Subject: Re: [Openstack] Ceph bootable cinder volume not working in Havana
>>>
>>> Could you give out your cinder.conf?
>>>
>>> On Tue, Dec 10, 2013 at 4:10 AM, Edward Lezondra
>>> <Edward.Lezondra at imc-chicago.com> wrote:
>>>> Hello,
>>>>
>>>>
>>>>
>>>> I’m having trouble getting a bootable cinder volume to work in Havana
>>>> using Ceph. I managed to get glance working correctly. I got cinder
>>>> volume to work, but once I try to create bootable cinder volume to an
>>>> image it errors out.
>>>>
>>>> My cinder.conf reflect exactly what the ceph docs tell one to do to
>>>> get it working. I was able to get this to work in grizzly using the
>>>> CEPH_ARGS="--id volumes" statement on the volume service, but the docs
>>>> and as a thread in the mailing list who got his to work didn’t use the
>>>> CEPH_ARGS="--id volumes" anymore for Havana. Judging from the logs I
>>>> have
>>>> permission issue on the rbd image. I made sure my keyrings have the
>>>> proper ownership and gave full permissions to everyone to them.
>>>>
>>>> My logs on cinder/volume shows the following:
>>>>
>>>>
>>>>
>>>> 2013-12-09 12:29:12.377 8468 ERROR cinder.volume.drivers.rbd
>>>> [req-75308325-0b69-41f9-8099-1c9d64c025b6
>>>> ce3265c65e304ad3b018a9fba063df88 0c3cbff39d97448d8cff1816793aea94]
>>>> error opening rbd image
>>>> 9e967bd7-ed6c-41bc-932d-1157bd72b778
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd Traceback
>>>> (most recent call last):
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 209, in __init__
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd
>>>> read_only=read_only)
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd File
>>>> "/usr/lib/python2.7/site-packages/rbd.py", line 351, in __init__
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd raise
>>>> make_ex(ret, 'error opening image %s at snapshot %s' % (name,
>>>> snapshot))
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd
>>>> PermissionError: error opening image
>>>> 9e967bd7-ed6c-41bc-932d-1157bd72b778 at snapshot snap
>>>>
>>>> 2013-12-09 12:29:12.377 8468 TRACE cinder.volume.drivers.rbd
>>>>
>>>> 2013-12-09 12:29:12.380 8468 WARNING cinder.taskflow.utils [-]
>>>> Activating 5 rollbacks due to <cinder.taskflow.utils.FlowFailure object
>>>> at 0x328c690>.
>>>>
>>>> 2013-12-09 12:29:12.485 8468 ERROR cinder.volume.flows.create_volume
>>>> [req-75308325-0b69-41f9-8099-1c9d64c025b6
>>>> ce3265c65e304ad3b018a9fba063df88 0c3cbff39d97448d8cff1816793aea94]
>>>> Unexpected build error:
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> Traceback (most recent call last):
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow
>>>> .py",
>>>> line 172, in run_it
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> result = runner(context, *args, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/utils.py", line 260,
>>>> in __call__
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> self.result = self.task(*args, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__
>>>> init__.py",
>>>> line 1499, in __call__
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> **volume_spec)
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__
>>>> init__.py",
>>>> line 1441, in _create_from_image
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> volume_ref, image_location, image_id)
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 729, in clone_image
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> if
>>>> image_location is None or not self._is_cloneable(image_location):
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 720, in _is_cloneable
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> read_only=True):
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 213, in __init__
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> raise
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>> TypeError: exceptions must be old-style classes or derived from
>>>> BaseException, not NoneType
>>>>
>>>> 2013-12-09 12:29:12.485 8468 TRACE cinder.volume.flows.create_volume
>>>>
>>>> 2013-12-09 12:29:12.486 8468 ERROR cinder.openstack.common.rpc.amqp
>>>> [req-75308325-0b69-41f9-8099-1c9d64c025b6
>>>> ce3265c65e304ad3b018a9fba063df88 0c3cbff39d97448d8cff1816793aea94]
>>>> Exception during message handling
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> Traceback (most recent call last):
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/openstack/common/rpc/amqp.py"
>>>> , line 441, in _process_data
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> **args)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/openstack/common/rpc/dispatch
>>>> er.py",
>>>> line 148, in dispatch
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> return getattr(proxyobj, method)(ctxt, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/utils.py", line 808, in
>>>> wrapper
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> return func(self, *args, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 257,
>>>> in create_volume
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> flow.run(context.elevated())
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/decorators.py", line
>>>> 105, in wrapper
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> return f(self, *args, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow
>>>> .py",
>>>> line 232, in run
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> run_it(r)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow
>>>> .py",
>>>> line 212, in run_it
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> self.rollback(context, cause)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> self.gen.next()
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/patterns/linear_flow
>>>> .py",
>>>> line 172, in run_it
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> result = runner(context, *args, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/taskflow/utils.py", line 260,
>>>> in __call__
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> self.result = self.task(*args, **kwargs)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__
>>>> init__.py",
>>>> line 1499, in __call__
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> **volume_spec)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/flows/create_volume/__
>>>> init__.py",
>>>> line 1441, in _create_from_image
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> volume_ref, image_location, image_id)
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 729, in clone_image
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> if
>>>> image_location is None or not self._is_cloneable(image_location):
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 720, in _is_cloneable
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> read_only=True):
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> File
>>>> "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line
>>>> 213, in __init__
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> raise
>>>>
>>>> 2013-12-09 12:29:12.486 8468 TRACE cinder.openstack.common.rpc.amqp
>>>> TypeError: exceptions must be old-style classes or derived from
>>>> BaseException, not NoneType
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________
>>>>
>>>> The information in this e-mail is intended only for the person or
>>>> entity to which it is addressed.
>>>>
>>>> It may contain confidential and /or privileged material. If someone
>>>> other than the intended recipient should receive this e-mail, he / she
>>>> shall not be entitled to read, disseminate, disclose or duplicate it.
>>>>
>>>> If you receive this e-mail unintentionally, please inform us
>>>> immediately by "reply" and then delete it from your system. Although
>>>> this information has been compiled with great care, neither IMC
>>>> Financial Markets & Asset Management nor any of its related entities
>>>> shall accept any responsibility for any errors, omissions or other
>>>> inaccuracies in this information or for the consequences thereof, nor
>>>> shall it be bound in any way by the contents of this e-mail or its
>>>> attachments. In the event of incomplete or incorrect transmission,
>>>> please return the e-mail to the sender and permanently delete this
>>>> message and any attachments.
>>>>
>>>> Messages and attachments are scanned for all known viruses. Always
>>>> scan attachments before opening them.
>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>
>>>
>>> --
>>> Best Regards,
>>>
>>> Wheat
>>>
>>> ________________________________
>>>
>>> The information in this e-mail is intended only for the person or entity
>>> to which it is addressed.
>>>
>>> It may contain confidential and /or privileged material. If someone other
>>> than the intended recipient should receive this e-mail, he / she shall not
>>> be entitled to read, disseminate, disclose or duplicate it.
>>>
>>> If you receive this e-mail unintentionally, please inform us immediately
>>> by "reply" and then delete it from your system. Although this information
>>> has been compiled with great care, neither IMC Financial Markets & Asset
>>> Management nor any of its related entities shall accept any responsibility
>>> for any errors, omissions or other inaccuracies in this information or for
>>> the consequences thereof, nor shall it be bound in any way by the contents
>>> of this e-mail or its attachments. In the event of incomplete or incorrect
>>> transmission, please return the e-mail to the sender and permanently delete
>>> this message and any attachments.
>>>
>>> Messages and attachments are scanned for all known viruses. Always scan
>>> attachments before opening them.
>>> _______________________________________________
>>> Mailing list:
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to : openstack at lists.openstack.org
>>> Unsubscribe :
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>> --
>> Alvin Starr || voice: (905)513-7688
>> Netvel Inc. || Cell: (416)806-0133
>> alvin at netvel.net ||
>>
>
>
--
Alvin Starr || voice: (905)513-7688
Netvel Inc. || Cell: (416)806-0133
alvin at netvel.net ||
More information about the Openstack
mailing list