Open Stack

Josh,

Thanks so much for the pointers.  I did not figure out that libvirt was
trying to read /etc/ceph/ceph.conf.  It was in fact being denied by
apparmor.  I fixed my problem by editing /etc/apparmor.d/libvirt/TEMPLATE
and adding "/etc/ceph/** r," to the template.  Now it works for newly
spawned VMs.

Again, thanks a lot.

 - Travis

On Fri, Jul 13, 2012 at 7:40 PM, Josh Durgin <josh.durgin at inktank.com>wrote:

> On 07/13/2012 03:09 PM, Travis Rhoden wrote:
>
>> Was this problem ever resolved?  I'm having the identical issue right now
>> (perhaps because I am following Sebastien's guide [0]).  I found this
>> question raised on LP [1], but no helpful responses, and no linked bug.  I
>> can confirm that I can attach an RBD instance manually through virsh, no
>> problem. So it is not a libvirt thing.  It sure appears to be a faulty
>> disk
>> attachment definition from OpenStack.
>>
>
> I didn't notice this launchpad question before, but I'll copy my answer
> below for the mailing list archives:
>
> Right now the RBD driver in OpenStack requires you to have your monitor
> addresses in /etc/ceph/ceph.conf. When you have them there, putting them in
> the libvirt xml is unnecessary. You do have to make sure that the unix user
> who ends up running qemu/kvm has read access to /etc/ceph/ceph.conf, and
> e.g. apparmor isn't preventing qemu/kvm from reading it.
>
> This dependency on /etc/ceph/ceph.conf should be removed in the future.
>
>
>  I'm running the latest packages on Ubuntu 12.04 for OpensStack (essex) and
>> Ceph (argonaut).
>>
>> I can confirm when using nova volume-attach it does not matter what device
>> you pick, dev/vdb, dev/vdf, whatever.  Always fails.  /dev/vdb manually
>> works great.
>>
>
> If the answer above doesn't help, what error message do you get?
> Is there any info in the libvirt logs for the instance?
>
> Josh
>
>  [0]
>> http://www.sebastien-han.fr/**blog/2012/06/10/introducing-**
>> ceph-to-openstack/<http://www.sebastien-han.fr/blog/2012/06/10/introducing-ceph-to-openstack/>
>> [1] https://answers.launchpad.net/**nova/+question/201366<https://answers.launchpad.net/nova/+question/201366>
>>
>>   - Travis
>>
>> On Fri, May 25, 2012 at 8:32 PM, Josh Durgin <josh.durgin at inktank.com>**
>> wrote:
>>
>>  On 05/25/2012 01:31 AM, Sébastien Han wrote:
>>>
>>>  Hi everyone,
>>>>
>>>> I setup a ceph cluster and I use the RBD driver for nova-volume.
>>>> I can create volumes and snapshots but currently I can't attach them to
>>>> an instance.
>>>> Apparently the volume is detected as busy but it's not, no matter which
>>>> name I choose.
>>>> I tried from horizon and the command line, same issue.
>>>>
>>>> $ nova --debug volume-attach de3743be-69be-4ec3-8de3-****fc7df6903a97
>>>> 12
>>>>
>>>> /dev/vdc
>>>> connect: (localhost, 5000)
>>>> send: 'POST /v2.0/tokens HTTP/1.1\r\nHost:
>>>> localhost:5000\r\nContent-****Length: 100\r\ncontent-type:
>>>> application/json\r\naccept-****encoding: gzip, deflate\r\naccept:
>>>> application/json\r\nuser-****agent: python-novaclient\r\n\r\n{"****
>>>> auth":
>>>>
>>>> {"tenantName": "admin", "passwordCredentials": {"username": "admin",
>>>> "password": "admin"}}}'
>>>> reply: 'HTTP/1.1 200 OK\r\n'
>>>> header: Content-Type: application/json
>>>> header: Vary: X-Auth-Token
>>>> header: Date: Fri, 25 May 2012 08:29:46 GMT
>>>> header: Transfer-Encoding: chunked
>>>> connect: (172.17.1.6, 8774)
>>>> send: u'GET
>>>> /v2/****d1f5d27ccf594cdbb034c8a4123494****e9/servers/de3743be-69be-**
>>>> 4ec3-**
>>>>
>>>> 8de3-fc7df6903a97
>>>> HTTP/1.1\r\nHost: 172.17.1.6:8774
>>>> <http://172.17.1.6:8774>\r\nx-****auth-project-id:
>>>> admin\r\nx-auth-token:
>>>>
>>>> 37ff6354b9114178889128175494c6****66\r\naccept-encoding: gzip,
>>>> deflate\r\naccept: application/json\r\nuser-****agent:
>>>>
>>>> python-novaclient\r\n\r\n'
>>>> reply: 'HTTP/1.1 200 OK\r\n'
>>>> header: X-Compute-Request-Id: req-fc7686dd-7dff-4ea2-b2dd-****
>>>> be584dadb7c1
>>>>
>>>> header: Content-Type: application/json
>>>> header: Content-Length: 1371
>>>> header: Date: Fri, 25 May 2012 08:29:46 GMT
>>>> send: u'POST
>>>> /v2/****d1f5d27ccf594cdbb034c8a4123494****e9/servers/de3743be-69be-**
>>>> 4ec3-**
>>>> 8de3-fc7df6903a97/os-volume_****attachments
>>>>
>>>> HTTP/1.1\r\nHost: 172.17.1.6:8774
>>>> <http://172.17.1.6:8774>\r\****nContent-Length:
>>>> 60\r\nx-auth-project-id:
>>>>
>>>>
>>>> admin\r\naccept-encoding: gzip, deflate\r\naccept:
>>>> application/json\r\nx-auth-****token:
>>>> 37ff6354b9114178889128175494c6****66\r\nuser-agent:
>>>> python-novaclient\r\ncontent-****type:
>>>> application/json\r\n\r\n{"****volumeAttachment": {"device": "/dev/vdc",
>>>>
>>>> "volumeId": 12}}'
>>>> reply: 'HTTP/1.1 200 OK\r\n'
>>>> header: X-Compute-Request-Id: req-4a22f603-4aad-4ae9-99a2-****
>>>> 6a7bcacf974f
>>>>
>>>> header: Content-Type: application/json
>>>> header: Content-Length: 48
>>>> header: Date: Fri, 25 May 2012 08:29:46 GMT
>>>> $ echo $?
>>>> 0
>>>>
>>>> See the logs here:
>>>> http://paste.openstack.org/****show/18163/<http://paste.openstack.org/**show/18163/>
>>>> <http://paste.**openstack.org/show/18163/<http://paste.openstack.org/show/18163/>
>>>> >
>>>>
>>>>
>>> Hi Seb,
>>>
>>> It looks like the DeviceIsBusy exception is raised when libvirt
>>> returns failure for any reason when attaching a volume. Try turning on
>>> debug logging for libvirt and attaching again, and posting the logs
>>> from libvirt.
>>>
>>> You might need to upgrade libvirt - the version in Ubuntu 12.04 should
>>> work, or upstream libvirt 0.9.12.
>>>
>>> Josh
>>>
>>>
>>>   If you need more information let me know :)
>>>
>>>> Thank you in advance.
>>>>
>>>> Cheers.
>>>>
>>>> ~Seb
>>>>
>>>>
>>> ______________________________****_________________
>>> Mailing list: https://launchpad.net/~****openstack<https://launchpad.net/%7E**openstack>
>>> <https://launchpad.**net/%7Eopenstack<https://launchpad.net/%7Eopenstack>
>>> >
>>> Post to     : openstack at lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~****openstack<https://launchpad.net/%7E**openstack>
>>> <https://launchpad.**net/%7Eopenstack<https://launchpad.net/%7Eopenstack>
>>> >
>>> More help   : https://help.launchpad.net/****ListHelp<https://help.launchpad.net/**ListHelp>
>>> <https://help.**launchpad.net/ListHelp<https://help.launchpad.net/ListHelp>
>>> >
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120713/e09513c1/attachment.html>