Hi again,

I did not manage to make it work, I cannot figure out how to connect all the pieces.

pollsters.d/octavia.yaml  https://paste.xinu.at/DERxh1/

pipeline.yaml https://paste.xinu.at/u1E42/

polling.yaml https://paste.xinu.at/MZWNs/

gnocchi_resources.yaml https://paste.xinu.at/j3AX/

gnocchi_client.py in resources_update_operations  https://paste.xinu.at/no5/ 

gnocchi resource-type show https://paste.xinu.at/7mZIyZ/

Do you mind if you do a full example using  "dynamic.network.services.vpn.connection" from https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html ?

Or maybe you can point me to the mistakes made in my configuration?

On Tue, Jul 7, 2020 at 2:43 PM Rafael Weingärtner <rafaelweingartner@gmail.com> wrote:

That is the right direction. I don't know why people hard-coded the initial pollsters' configs and did not document the relation between Gnocchi and Ceilometer properly. They (Ceilometer and Gnocchi) are not a single system, but interdependent systems to implement a monitoring solution. Ceilometer is the component that gathers data/information, processes, and then persists it somewhere. Gnocchi is one of the options that Ceilometer can use to persist data. By default, Ceilometer creates some basic configurations in Gnocchi to store data, such as some default resource-types with default attributes. However, we do not need (should not) rely on this default config.

You can create and use custom resources to fit the stack to your needs. This can be achieved via `gnocchi resource-type create -a <attribute_name>:<attribute_type>:<required> <name_of_resource_type>` and `gnocchi resource-type create -u <new_attribute_name>:<attribute_type>:<required> <name_of_resource_type>`. Then, in the `custom_gnocchi_resources.yaml` (if you use Kolla-ansible), you can customize the mapping of metrics to resource-types in Gnocchi.

On Tue, Jul 7, 2020 at 7:49 AM Ionut Biru <ionut@fleio.com> wrote:

Hello again,

What's the proper way to handle dynamic pollsters in gnocchi ?

Right now ceilometer returns:

WARNING ceilometer.publisher.gnocchi [-] metric dynamic.network.octavia is not handled by Gnocchi

I found https://docs.openstack.org/ceilometer/latest/contributor/new_resource_types.html but I'm not sure if is the right direction. 

On Tue, Jul 7, 2020 at 10:52 AM Ionut Biru <ionut@fleio.com> wrote:

Seems to work fine now. Thanks.

On Mon, Jul 6, 2020 at 8:12 PM Rafael Weingärtner <rafaelweingartner@gmail.com> wrote:

It looks like a coding error that we left behind during a major refactoring that we introduced upstream.

I created a patch for it. Can you check/review and test it?

https://review.opendev.org/739555

On Mon, Jul 6, 2020 at 11:17 AM Ionut Biru <ionut@fleio.com> wrote:

Hi Rafael,

I have an error and I cannot resolve it myself.

https://paste.xinu.at/LEfdXD/

Do you happen to know what's wrong?

endpoint list https://paste.xinu.at/v3j1jl/

octavia.yaml https://paste.xinu.at/TIxfOz/

polling.yaml https://paste.xinu.at/oBEFj/

pipeline.yaml https://paste.xinu.at/qvEdTX/

On Sat, Jul 4, 2020 at 1:10 AM Rafael Weingärtner <rafaelweingartner@gmail.com> wrote:

Good catch. I fixed the docs.

https://review.opendev.org/#/c/739288/

On Fri, Jul 3, 2020 at 1:59 PM Ionut Biru <ionut@fleio.com> wrote:

Hi,

I just noticed that the example dynamic.network.services.vpn.connection from https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html has the wrong indentation.

This https://paste.xinu.at/6PTfsM/ is loaded without any error.

Now I have to see why is not polling from it

On Fri, Jul 3, 2020 at 7:19 PM Ionut Biru <ionut@fleio.com> wrote:

Hi Rafael,

I think I applied all the reviews successfully but I tried to do an octavia dynamic poller but I have couples of errors.

Here is the octavia.yaml: https://paste.xinu.at/kDN6SV/

Error is about syntax error near name: https://paste.xinu.at/MHgDBY/

if i remove the - in front of name like this: https://paste.xinu.at/K7s5I8/

The error is different this time: https://paste.xinu.at/zWdC0U/

Is there something I missed or is something wrong  in yaml?

On Thu, Jul 2, 2020 at 5:50 PM Rafael Weingärtner <rafaelweingartner@gmail.com> wrote:

Since the merging window for ussuri was long passed for those commits, is it safe to assume that it will not land in stable/ussuri at all and those will be available for victoria?

I would say so. We are lacking people to review and then merge it.

How safe is to cherry pick those commits and use them in production?

As long as the person executing the cherry-picks, and maintaining the code knows what she/he is doing, you should be safe. The guys that are using this implementation (and others that I and my colleagues proposed), have a few openstack components that are customized with the patches/enhancements/extensions we developed so far; this means, they are not using the community version, but something in-between (the community releases + the patches we did). Of course, it is only possible, because we are the ones creating and maintaining these codes; therefore, we can assure quality for production.

On Thu, Jul 2, 2020 at 9:43 AM Ionut Biru <ionut@fleio.com> wrote:

Hello Rafael,

Since the merging window for ussuri was long passed for those commits, is it safe to assume that it will not land in stable/ussuri at all and those will be available for victoria?

How safe is to cherry pick those commits and use them in production?

On Fri, Apr 24, 2020 at 3:06 PM Rafael Weingärtner <rafaelweingartner@gmail.com> wrote:

The dynamic pollster in Ceilometer will be first released in Ussuri. However, there are some important PRs still waiting for a merge, that might be important for your use case:

* https://review.opendev.org/#/c/722092/

* https://review.opendev.org/#/c/715180/

* https://review.opendev.org/#/c/715289/

* https://review.opendev.org/#/c/679999/

* https://review.opendev.org/#/c/709807/

On Fri, Apr 24, 2020 at 8:18 AM Carlos Goncalves <cgoncalves@redhat.com> wrote:

On Fri, Apr 24, 2020 at 12:20 PM Ionut Biru <ionut@fleio.com> wrote:

Hello,

I want to meter the loadbalancer into gnocchi for billing purposes in stein/train and ceilometer doesn't support dynamic pollsters. 

I think I misunderstood your use case, sorry. I read it as if you wanted to know "if a loadbalancer was deployed and has status active".

 

Until I upgrade to Ussuri, is there a way to accomplish this?

I'm not sure Ceilometer supports it even in Ussuri. I'll defer to the Ceilometer project.

 

On Fri, Apr 24, 2020 at 12:45 PM Carlos Goncalves <cgoncalves@redhat.com> wrote:

Hi Ionut,

On Fri, Apr 24, 2020 at 11:27 AM Ionut Biru <ionut@fleio.com> wrote:

Hello guys,

I was trying to add in polling.yaml and pipeline from ceilometer the following:

   - network.services.lb.active.connections
      - network.services.lb.health_monitor
      - network.services.lb.incoming.bytes
      - network.services.lb.listener
      - network.services.lb.loadbalancer
      - network.services.lb.member
      - network.services.lb.outgoing.bytes
      - network.services.lb.pool
      - network.services.lb.total.connections

But it doesn't work, I think they are for the old lbs that were supported in neutron.

I found https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html but this is not available in stein or train.

I was wondering if there is a way to meter loadbalancers from octavia. 

I mostly want for start to just meter if a loadbalancer was deployed and has status active.

You can get the provisioning and operating status of Octavia load balancers via the Octavia API. There is also an API endpoint that returns the full load balancer status tree [1]. Additionally, Octavia has three API endpoints for statistics [2][3][4].

I hope this helps with your use case.

Cheers,

Carlos

[1] https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-the-load-balancer-status-tree-detail#get-the-load-balancer-status-tree

[2] https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-load-balancer-statistics-detail#get-load-balancer-statistics

[3] https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-listener-statistics-detail#get-listener-statistics

[4] https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=show-amphora-statistics-detail#show-amphora-statistics

 

--

Ionut Biru - https://fleio.com

--

Ionut Biru - https://fleio.com

--

Rafael Weingärtner

--

Ionut Biru - https://fleio.com

--

Rafael Weingärtner

--

Ionut Biru - https://fleio.com

--

Ionut Biru - https://fleio.com

--

Rafael Weingärtner

--

Ionut Biru - https://fleio.com

--

Rafael Weingärtner

--

Ionut Biru - https://fleio.com

--

Ionut Biru - https://fleio.com

--

Rafael Weingärtner

--

Ionut Biru - https://fleio.com