[telemetry][cloudkitty][ceilometer] Billing windows instances
Hi there! We're using telemetry+cloudkitty for our rating. We're happy of it, though we're having the issue that we would like to bill instances running Windows. The example given in the Cloudkitty doc shows how to bill more when an instance runs Windows. That works in theory, though in practice, Microsoft has a license model based on how many vCPU the instannce runs. So the model that the Cloudkitty documentation shows simply doesn't work with the SPLA thingy. We've looked at options. One way would of course writing a new custom pollster, but we don't like the idea: this would mean polling for all of the thousands of instances that are running in our deployments. So we would very much prefer having ceilometer running on compute node (with polling_namespaces=compute) to do the work, as this scales a way better. However, Ceilometer polls libvirt, which only has the information about the image ID, not the metadata associated with the image (like the property os_type=windows, for example). So, is there a better way than a dynamic pollster? Can this be done with ceilometer on the compute nodes? Cheers, Thomas Goirand (zigo)
Hello Zigo!, You might want to take a look at the new implementations we made in ceilometer, and CloudKitty. - https://review.opendev.org/c/openstack/cloudkitty/+/861806 - https://review.opendev.org/c/openstack/ceilometer/+/856178 - https://review.opendev.org/c/openstack/ceilometer/+/852021 - https://review.opendev.org/c/openstack/ceilometer/+/850253 - https://review.opendev.org/c/openstack/ceilometer/+/855953 Not directly relate to this use case, but also might interest you: - https://review.opendev.org/c/openstack/cloudkitty/+/861786 - https://review.opendev.org/c/openstack/cloudkitty/+/861807 - https://review.opendev.org/c/openstack/cloudkitty/+/861908 - https://review.opendev.org/c/openstack/ceilometer/+/856972 - https://review.opendev.org/c/openstack/ceilometer/+/861109 - https://review.opendev.org/c/openstack/ceilometer/+/856304 - https://review.opendev.org/c/openstack/ceilometer/+/856305 In short, we can now create Ceilometer compute dynamic pollsters, which can execute scripts in the host, and check the actual operating system installed in the VM. Then, this data can be pushed back to the storage backend via Ceilometer as an attribute, which is then processed in CloudKitty. Furthermore, we extended cloudkitty to generate different ratings for the same metric. Therefore, by doing this, we do not need multiple metrics to have different CloudKitty ratings appearing for users. This allows us, for instance, to have one rating for the VM usage itself, and others for each license, and so on. On Fri, Oct 21, 2022 at 5:19 AM Thomas Goirand <zigo@debian.org> wrote:
Hi there!
We're using telemetry+cloudkitty for our rating. We're happy of it, though we're having the issue that we would like to bill instances running Windows.
The example given in the Cloudkitty doc shows how to bill more when an instance runs Windows. That works in theory, though in practice, Microsoft has a license model based on how many vCPU the instannce runs. So the model that the Cloudkitty documentation shows simply doesn't work with the SPLA thingy.
We've looked at options. One way would of course writing a new custom pollster, but we don't like the idea: this would mean polling for all of the thousands of instances that are running in our deployments. So we would very much prefer having ceilometer running on compute node (with polling_namespaces=compute) to do the work, as this scales a way better.
However, Ceilometer polls libvirt, which only has the information about the image ID, not the metadata associated with the image (like the property os_type=windows, for example).
So, is there a better way than a dynamic pollster? Can this be done with ceilometer on the compute nodes?
Cheers,
Thomas Goirand (zigo)
-- Rafael Weingärtner
On 10/21/22 17:40, Rafael Weingärtner wrote:
Hello Zigo!, You might want to take a look at the new implementations we made in ceilometer, and CloudKitty. - https://review.opendev.org/c/openstack/cloudkitty/+/861806 <https://review.opendev.org/c/openstack/cloudkitty/+/861806> - https://review.opendev.org/c/openstack/ceilometer/+/856178 <https://review.opendev.org/c/openstack/ceilometer/+/856178> - https://review.opendev.org/c/openstack/ceilometer/+/852021 <https://review.opendev.org/c/openstack/ceilometer/+/852021> - https://review.opendev.org/c/openstack/ceilometer/+/850253 <https://review.opendev.org/c/openstack/ceilometer/+/850253> - https://review.opendev.org/c/openstack/ceilometer/+/855953 <https://review.opendev.org/c/openstack/ceilometer/+/855953>
Not directly relate to this use case, but also might interest you: - https://review.opendev.org/c/openstack/cloudkitty/+/861786 <https://review.opendev.org/c/openstack/cloudkitty/+/861786> - https://review.opendev.org/c/openstack/cloudkitty/+/861807 <https://review.opendev.org/c/openstack/cloudkitty/+/861807> - https://review.opendev.org/c/openstack/cloudkitty/+/861908 <https://review.opendev.org/c/openstack/cloudkitty/+/861908> - https://review.opendev.org/c/openstack/ceilometer/+/856972 <https://review.opendev.org/c/openstack/ceilometer/+/856972> - https://review.opendev.org/c/openstack/ceilometer/+/861109 <https://review.opendev.org/c/openstack/ceilometer/+/861109> - https://review.opendev.org/c/openstack/ceilometer/+/856304 <https://review.opendev.org/c/openstack/ceilometer/+/856304> - https://review.opendev.org/c/openstack/ceilometer/+/856305 <https://review.opendev.org/c/openstack/ceilometer/+/856305>
In short, we can now create Ceilometer compute dynamic pollsters, which can execute scripts in the host, and check the actual operating system installed in the VM. Then, this data can be pushed back to the storage backend via Ceilometer as an attribute, which is then processed in CloudKitty. Furthermore, we extended cloudkitty to generate different ratings for the same metric. Therefore, by doing this, we do not need multiple metrics to have different CloudKitty ratings appearing for users. This allows us, for instance, to have one rating for the VM usage itself, and others for each license, and so on.
Hi Raphael! Thanks a lot for all of the above (both the patches merged upstream themselves, and taking the time to give me reference to them). We've backported this to our production version (ie: Victoria) without too much pain (it took me like 4 hours to do so, cherry-picking missing patches on top of which these were applied). We then wrote a quick command to produce a JSON containing the image type (as reported by the os_type property of the image) that we put in cache in the compute, and then dump this image type, and the associated project ID. This looks promising: we only need to write the dynamic pollster now! :) So really, thanks a lot. I'll let you know when we have a full solution (that I will also publish as free software). Cheers, Thomas Goirand (zigo)
Awesome! i am glad to hear that you guys are managing to use it. If you need anything, please let me know. On Mon, Oct 24, 2022 at 11:21 AM Thomas Goirand <zigo@debian.org> wrote:
On 10/21/22 17:40, Rafael Weingärtner wrote:
Hello Zigo!, You might want to take a look at the new implementations we made in ceilometer, and CloudKitty. - https://review.opendev.org/c/openstack/cloudkitty/+/861806 <https://review.opendev.org/c/openstack/cloudkitty/+/861806> - https://review.opendev.org/c/openstack/ceilometer/+/856178 <https://review.opendev.org/c/openstack/ceilometer/+/856178> - https://review.opendev.org/c/openstack/ceilometer/+/852021 <https://review.opendev.org/c/openstack/ceilometer/+/852021> - https://review.opendev.org/c/openstack/ceilometer/+/850253 <https://review.opendev.org/c/openstack/ceilometer/+/850253> - https://review.opendev.org/c/openstack/ceilometer/+/855953 <https://review.opendev.org/c/openstack/ceilometer/+/855953>
Not directly relate to this use case, but also might interest you: - https://review.opendev.org/c/openstack/cloudkitty/+/861786 <https://review.opendev.org/c/openstack/cloudkitty/+/861786> - https://review.opendev.org/c/openstack/cloudkitty/+/861807 <https://review.opendev.org/c/openstack/cloudkitty/+/861807> - https://review.opendev.org/c/openstack/cloudkitty/+/861908 <https://review.opendev.org/c/openstack/cloudkitty/+/861908> - https://review.opendev.org/c/openstack/ceilometer/+/856972 <https://review.opendev.org/c/openstack/ceilometer/+/856972> - https://review.opendev.org/c/openstack/ceilometer/+/861109 <https://review.opendev.org/c/openstack/ceilometer/+/861109> - https://review.opendev.org/c/openstack/ceilometer/+/856304 <https://review.opendev.org/c/openstack/ceilometer/+/856304> - https://review.opendev.org/c/openstack/ceilometer/+/856305 <https://review.opendev.org/c/openstack/ceilometer/+/856305>
In short, we can now create Ceilometer compute dynamic pollsters, which can execute scripts in the host, and check the actual operating system installed in the VM. Then, this data can be pushed back to the storage backend via Ceilometer as an attribute, which is then processed in CloudKitty. Furthermore, we extended cloudkitty to generate different ratings for the same metric. Therefore, by doing this, we do not need multiple metrics to have different CloudKitty ratings appearing for users. This allows us, for instance, to have one rating for the VM usage itself, and others for each license, and so on.
Hi Raphael!
Thanks a lot for all of the above (both the patches merged upstream themselves, and taking the time to give me reference to them).
We've backported this to our production version (ie: Victoria) without too much pain (it took me like 4 hours to do so, cherry-picking missing patches on top of which these were applied). We then wrote a quick command to produce a JSON containing the image type (as reported by the os_type property of the image) that we put in cache in the compute, and then dump this image type, and the associated project ID. This looks promising: we only need to write the dynamic pollster now! :)
So really, thanks a lot. I'll let you know when we have a full solution (that I will also publish as free software).
Cheers,
Thomas Goirand (zigo)
-- Rafael Weingärtner
participants (2)
-
Rafael Weingärtner
-
Thomas Goirand