[openstack-dev] [Ceph][ceilometer][libvirt] Libvirt error during instance disk allocation metering

ZhiQiang Fan aji.zqfan at gmail.com
Sat Apr 30 00:59:20 UTC 2016


Hi Ceph devs,

I raise this bug again because as Ceph becomes more popular, our customer
suffers from it and I have no solution for it.

*Is there anyway to get the usage of a disk which is a Ceph volume? (Not
sure if the term is right or not)*

Ceilometer use libvirt.domain.blockInfo(device) to get the usage
(allocation/physical/capacity) of a disk, [1]. This works fine when the VM
is boot from local file system, and according to the note from [2], it
seems blockInfo uses stat instead of querying qemu, so **maybe** this is
why it doesn't work for the network type disk: internal error: missing
storage backend for network files using rbd protocol.

libvirt.domain.blockStats() doesn't return usage but r/w bytes/requests,
 neither domainListGetStats

I haven't try virStorageVolInfo yet, because don't know the args, but it
doesn't return all the 3 usage dimension of a disk, only capacity and
allocation[3]

So I'm asking if anyone can help me to resolve this issue.
Thank you very much!

PS: change subject to get ceph devs noticed.

[1] https://review.openstack.org/#/c/145819/
[2]  https://www.redhat.com/archives/libvir-list/2014-December/msg00762.html

[3] https://libvirt.org/html/libvirt-libvirt-storage.html#virStorageVolInfo
[4] https://bugs.launchpad.net/ceilometer/+bug/1457440

On Wed, Nov 18, 2015 at 11:46 PM, Ilya Tyaptin <ityaptin at mirantis.com>
wrote:

> Hi, folks!
>
> In our deployed envs we met with a libvirt error *"missing storage
> backend for network files using rbd protocol"* in *virDomainGetBlockInfo*
>  call [1] <http://paste.openstack.org/show/479237/>
> .
> This exception is raised when Ceilometer are trying to get info about VM
> disk usage and allocation.
> It only affects getting measures for a some disk pollsters which added in
> this CR [2]
> <https://review.openstack.org/#/c/145819/23/ceilometer/compute/virt/libvirt/inspector.py,cm>
>  with specified libvirt call [3]
> <https://github.com/openstack/ceilometer/blob/stable/liberty/ceilometer/compute/virt/libvirt/inspector.py#L215>
>  .
> These pollsters have been added in the Kilo cycle and successful work in
> Kilo deployments, but it doesn't work now.
>
> Also, we have a bug in the upstream launchpad [4]
> <https://bugs.launchpad.net/ceilometer/+bug/1457440>
>  but it have not been fixed yet.
>
> I would glad to see any ideas about root cause of this issue or ways to
> fixing it.
>
> Thank you in advance!
>
> References:
> [1] Traceback <http://paste.openstack.org/show/479237/>
>
>
> ./ceilometer-polling.log.0:4192:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk Traceback (most recent call last):
> ./ceilometer-polling.log.0:4193:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk   File
> "/usr/lib/python2.7/dist-packages/ceilometer/compute/pollsters/disk.py",
> line 703, in get_samples
> ./ceilometer-polling.log.0:4194:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk     instance,
> ./ceilometer-polling.log.0:4195:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk   File
> "/usr/lib/python2.7/dist-packages/ceilometer/compute/pollsters/disk.py",
> line 672, in _populate_cache
> ./ceilometer-polling.log.0:4196:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk     for disk, info in disk_info:
> ./ceilometer-polling.log.0:4197:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk   File
> "/usr/lib/python2.7/dist-packages/ceilometer/compute/virt/libvirt/inspector.py",
> line 215, in inspect_disk_info
> ./ceilometer-polling.log.0:4198:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk     block_info = domain.blockInfo(device)
> ./ceilometer-polling.log.0:4199:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk   File
> "/usr/lib/python2.7/dist-packages/libvirt.py", line 658, in blockInfo
> ./ceilometer-polling.log.0:4200:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk     if ret is None: raise libvirtError
> ('virDomainGetBlockInfo() failed', dom=self)
> ./ceilometer-polling.log.0:4201:2015-11-17 16:20:54.807 14107 ERROR
> ceilometer.compute.pollsters.disk libvirtError: internal error: missing
> storage backend for network files using rbd protocol
>
> [2] CR with this commit:
> https://review.openstack.org/#/c/145819/23/ceilometer/compute/virt/libvirt/inspector.py,cm
>
> [3] Code entry:
> https://github.com/openstack/ceilometer/blob/stable/liberty/ceilometer/compute/virt/libvirt/inspector.py#L215
> [4] Upstream bug: https://bugs.launchpad.net/ceilometer/+bug/1457440
>
>
> Best regards,
>
> Tyaptin Il​y​a,
>
> Ceilometer developer,
>
> Mirantis Inc.
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160430/6830f3a0/attachment.html>


More information about the OpenStack-dev mailing list