[ceilometer][octavia] polling meters

Ionut Biru ionut at fleio.com
Tue Jul 14 14:00:52 UTC 2020


Hi,

Thanks for the information.

I made it work by using only one attribute at that time the error was
something related to the type of an attribute and I got rid of it.

On Fri, Jul 10, 2020 at 8:24 PM Rafael Weingärtner <
rafaelweingartner at gmail.com> wrote:

> Sure, this is a minimalistic config I used for testing (watch for the
> indentation issues that might happen due to copy/paste into Gmail).
>
>> cat ceilometer/pollsters.d/vpn-connection-dynamic-pollster.yaml
>> ---
>>
>> - name: "dynamic_pollster.network.services.vpn.connection"
>>   sample_type: "gauge"
>>   unit: "ipsec_site_connection"
>>   value_attribute: "status"
>>   endpoint_type: "network"
>>   url_path: "v2.0/vpn/ipsec-site-connections"
>>   metadata_fields:
>>     - "name"
>>     - "vpnservice_id"
>>     - "description"
>>     - "status"
>>     - "peer_address"
>>   value_mapping:
>>     ACTIVE: "1"
>>     DOWN: "0"
>>   metadata_mapping:
>>     name: "display_name"
>>   default_value: 0
>>
>
> Then, the polling.yaml file
>
> cat ceilometer/polling.yaml | grep -A 3 vpnass
>>     - name: vpnass_pollsters
>>       interval: 600
>>       meters:
>>         - dynamic_pollster.network.services.vpn.connection
>>
>
> And last, but not least, the custom_gnocchi_resources file.
>
>> cat ceilometer/custom_gnocchi_resources.yaml | grep -B 2 -A 9
>> "dynamic_pollster.network.services.vpn.connection"
>>   - resource_type: s2svpn
>>     metrics:
>>       dynamic_pollster.network.services.vpn.connection:
>>     attributes:
>>       name: resource_metadata.name
>>       vpnservice_id: resource_metadata.vpnservice_id
>>       description: resource_metadata.description
>>       status: resource_metadata.status
>>       peer_address: resource_metadata.peer_address
>>       display_name: resource_metadata.display_name
>>
>
> Bear in mind that you need to create the Gnocchi resource type.
>
>> gnocchi resource-type show s2svpn
>>
>> +--------------------------+-----------------------------------------------------------+
>> | Field                    | Value
>>               |
>>
>> +--------------------------+-----------------------------------------------------------+
>> | attributes/description   | max_length=255, min_length=0,
>> required=False, type=string |
>> | attributes/display_name  | max_length=255, min_length=0,
>> required=False, type=string |
>> | attributes/name          | max_length=255, min_length=0,
>> required=False, type=string |
>> | attributes/peer_address  | max_length=255, min_length=0,
>> required=False, type=string |
>> | attributes/status        | max_length=255, min_length=0,
>> required=False, type=string |
>> | attributes/vpnservice_id | required=False, type=uuid
>>               |
>> | name                     | s2svpn
>>              |
>> | state                    | active
>>              |
>>
>> +--------------------------+-----------------------------------------------------------+
>>
>
> What is the problem you are having?
>
> On Fri, Jul 10, 2020 at 10:50 AM Ionut Biru <ionut at fleio.com> wrote:
>
>> Hi again,
>>
>> I did not manage to make it work, I cannot figure out how to connect all
>> the pieces.
>>
>> pollsters.d/octavia.yaml  https://paste.xinu.at/DERxh1/
>> pipeline.yaml https://paste.xinu.at/u1E42/
>> polling.yaml https://paste.xinu.at/MZWNs/
>> gnocchi_resources.yaml https://paste.xinu.at/j3AX/
>> gnocchi_client.py in resources_update_operations
>> https://paste.xinu.at/no5/
>> gnocchi resource-type show https://paste.xinu.at/7mZIyZ/
>> Do you mind if you do a full example
>> using  "dynamic.network.services.vpn.connection" from
>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html
>>  ?
>>
>> Or maybe you can point me to the mistakes made in my configuration?
>>
>>
>> On Tue, Jul 7, 2020 at 2:43 PM Rafael Weingärtner <
>> rafaelweingartner at gmail.com> wrote:
>>
>>> That is the right direction. I don't know why people hard-coded the
>>> initial pollsters' configs and did not document the relation between
>>> Gnocchi and Ceilometer properly. They (Ceilometer and Gnocchi) are not a
>>> single system, but interdependent systems to implement a monitoring
>>> solution. Ceilometer is the component that gathers data/information,
>>> processes, and then persists it somewhere. Gnocchi is one of the options
>>> that Ceilometer can use to persist data. By default, Ceilometer creates
>>> some basic configurations in Gnocchi to store data, such as some default
>>> resource-types with default attributes. However, we do not need (should
>>> not) rely on this default config.
>>>
>>> You can create and use custom resources to fit the stack to your needs.
>>> This can be achieved via `gnocchi resource-type create -a
>>> <attribute_name>:<attribute_type>:<required> <name_of_resource_type>` and
>>> `gnocchi resource-type create -u
>>> <new_attribute_name>:<attribute_type>:<required> <name_of_resource_type>`.
>>> Then, in the `custom_gnocchi_resources.yaml` (if you use Kolla-ansible),
>>> you can customize the mapping of metrics to resource-types in Gnocchi.
>>>
>>> On Tue, Jul 7, 2020 at 7:49 AM Ionut Biru <ionut at fleio.com> wrote:
>>>
>>>> Hello again,
>>>>
>>>> What's the proper way to handle dynamic pollsters in gnocchi ?
>>>> Right now ceilometer returns:
>>>>
>>>> WARNING ceilometer.publisher.gnocchi [-] metric dynamic.network.octavia
>>>> is not handled by Gnocchi
>>>>
>>>> I found
>>>> https://docs.openstack.org/ceilometer/latest/contributor/new_resource_types.html
>>>> but I'm not sure if is the right direction.
>>>>
>>>> On Tue, Jul 7, 2020 at 10:52 AM Ionut Biru <ionut at fleio.com> wrote:
>>>>
>>>>> Seems to work fine now. Thanks.
>>>>>
>>>>> On Mon, Jul 6, 2020 at 8:12 PM Rafael Weingärtner <
>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>
>>>>>> It looks like a coding error that we left behind during a major
>>>>>> refactoring that we introduced upstream.
>>>>>> I created a patch for it. Can you check/review and test it?
>>>>>> https://review.opendev.org/739555
>>>>>>
>>>>>> On Mon, Jul 6, 2020 at 11:17 AM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>
>>>>>>> Hi Rafael,
>>>>>>>
>>>>>>> I have an error and I cannot resolve it myself.
>>>>>>>
>>>>>>> https://paste.xinu.at/LEfdXD/
>>>>>>>
>>>>>>> Do you happen to know what's wrong?
>>>>>>>
>>>>>>> endpoint list https://paste.xinu.at/v3j1jl/
>>>>>>> octavia.yaml https://paste.xinu.at/TIxfOz/
>>>>>>> polling.yaml https://paste.xinu.at/oBEFj/
>>>>>>> pipeline.yaml https://paste.xinu.at/qvEdTX/
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Jul 4, 2020 at 1:10 AM Rafael Weingärtner <
>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>
>>>>>>>> Good catch. I fixed the docs.
>>>>>>>> https://review.opendev.org/#/c/739288/
>>>>>>>>
>>>>>>>> On Fri, Jul 3, 2020 at 1:59 PM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I just noticed that the example
>>>>>>>>> dynamic.network.services.vpn.connection from
>>>>>>>>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html has
>>>>>>>>> the wrong indentation.
>>>>>>>>> This https://paste.xinu.at/6PTfsM/ is loaded without any error.
>>>>>>>>>
>>>>>>>>> Now I have to see why is not polling from it
>>>>>>>>>
>>>>>>>>> On Fri, Jul 3, 2020 at 7:19 PM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Rafael,
>>>>>>>>>>
>>>>>>>>>> I think I applied all the reviews successfully but I tried to do
>>>>>>>>>> an octavia dynamic poller but I have couples of errors.
>>>>>>>>>>
>>>>>>>>>> Here is the octavia.yaml: https://paste.xinu.at/kDN6SV/
>>>>>>>>>> Error is about syntax error near name:
>>>>>>>>>> https://paste.xinu.at/MHgDBY/
>>>>>>>>>>
>>>>>>>>>> if i remove the - in front of name like this:
>>>>>>>>>> https://paste.xinu.at/K7s5I8/
>>>>>>>>>> The error is different this time: https://paste.xinu.at/zWdC0U/
>>>>>>>>>>
>>>>>>>>>> Is there something I missed or is something wrong  in yaml?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 2, 2020 at 5:50 PM Rafael Weingärtner <
>>>>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Since the merging window for ussuri was long passed for those
>>>>>>>>>>>> commits, is it safe to assume that it will not land in stable/ussuri at all
>>>>>>>>>>>> and those will be available for victoria?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I would say so. We are lacking people to review and then merge
>>>>>>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> How safe is to cherry pick those commits and use them in
>>>>>>>>>>>> production?
>>>>>>>>>>>>
>>>>>>>>>>> As long as the person executing the cherry-picks, and
>>>>>>>>>>> maintaining the code knows what she/he is doing, you should be safe. The
>>>>>>>>>>> guys that are using this implementation (and others that I and my
>>>>>>>>>>> colleagues proposed), have a few openstack components that are customized
>>>>>>>>>>> with the patches/enhancements/extensions we developed so far; this means,
>>>>>>>>>>> they are not using the community version, but something in-between (the
>>>>>>>>>>> community releases + the patches we did). Of course, it is only possible,
>>>>>>>>>>> because we are the ones creating and maintaining these codes; therefore, we
>>>>>>>>>>> can assure quality for production.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 2, 2020 at 9:43 AM Ionut Biru <ionut at fleio.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Rafael,
>>>>>>>>>>>>
>>>>>>>>>>>> Since the merging window for ussuri was long passed for those
>>>>>>>>>>>> commits, is it safe to assume that it will not land in stable/ussuri at all
>>>>>>>>>>>> and those will be available for victoria?
>>>>>>>>>>>>
>>>>>>>>>>>> How safe is to cherry pick those commits and use them in
>>>>>>>>>>>> production?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Apr 24, 2020 at 3:06 PM Rafael Weingärtner <
>>>>>>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> The dynamic pollster in Ceilometer will be first released in
>>>>>>>>>>>>> Ussuri. However, there are some important PRs still waiting for a merge,
>>>>>>>>>>>>> that might be important for your use case:
>>>>>>>>>>>>> * https://review.opendev.org/#/c/722092/
>>>>>>>>>>>>> * https://review.opendev.org/#/c/715180/
>>>>>>>>>>>>> * https://review.opendev.org/#/c/715289/
>>>>>>>>>>>>> * https://review.opendev.org/#/c/679999/
>>>>>>>>>>>>> * https://review.opendev.org/#/c/709807/
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 8:18 AM Carlos Goncalves <
>>>>>>>>>>>>> cgoncalves at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 12:20 PM Ionut Biru <ionut at fleio.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I want to meter the loadbalancer into gnocchi for billing
>>>>>>>>>>>>>>> purposes in stein/train and ceilometer doesn't support dynamic pollsters.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think I misunderstood your use case, sorry. I read it as if
>>>>>>>>>>>>>> you wanted to know "if a loadbalancer was deployed and has status active".
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Until I upgrade to Ussuri, is there a way to accomplish this?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm not sure Ceilometer supports it even in Ussuri. I'll
>>>>>>>>>>>>>> defer to the Ceilometer project.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 12:45 PM Carlos Goncalves <
>>>>>>>>>>>>>>> cgoncalves at redhat.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Ionut,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 11:27 AM Ionut Biru <
>>>>>>>>>>>>>>>> ionut at fleio.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello guys,
>>>>>>>>>>>>>>>>> I was trying to add in polling.yaml and pipeline from
>>>>>>>>>>>>>>>>> ceilometer the following:
>>>>>>>>>>>>>>>>>    - network.services.lb.active.connections
>>>>>>>>>>>>>>>>>       - network.services.lb.health_monitor
>>>>>>>>>>>>>>>>>       - network.services.lb.incoming.bytes
>>>>>>>>>>>>>>>>>       - network.services.lb.listener
>>>>>>>>>>>>>>>>>       - network.services.lb.loadbalancer
>>>>>>>>>>>>>>>>>       - network.services.lb.member
>>>>>>>>>>>>>>>>>       - network.services.lb.outgoing.bytes
>>>>>>>>>>>>>>>>>       - network.services.lb.pool
>>>>>>>>>>>>>>>>>       - network.services.lb.total.connections
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But it doesn't work, I think they are for the old lbs that
>>>>>>>>>>>>>>>>> were supported in neutron.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I found
>>>>>>>>>>>>>>>>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html
>>>>>>>>>>>>>>>>> but this is not available in stein or train.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I was wondering if there is a way to meter
>>>>>>>>>>>>>>>>> loadbalancers from octavia.
>>>>>>>>>>>>>>>>> I mostly want for start to just meter if a loadbalancer
>>>>>>>>>>>>>>>>> was deployed and has status active.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You can get the provisioning and operating status of
>>>>>>>>>>>>>>>> Octavia load balancers via the Octavia API. There is also an API endpoint
>>>>>>>>>>>>>>>> that returns the full load balancer status tree [1].
>>>>>>>>>>>>>>>> Additionally, Octavia has three API endpoints for
>>>>>>>>>>>>>>>> statistics [2][3][4].
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I hope this helps with your use case.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>> Carlos
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-the-load-balancer-status-tree-detail#get-the-load-balancer-status-tree
>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-load-balancer-statistics-detail#get-load-balancer-statistics
>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-listener-statistics-detail#get-listener-statistics
>>>>>>>>>>>>>>>> [4]
>>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=show-amphora-statistics-detail#show-amphora-statistics
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Rafael Weingärtner
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Rafael Weingärtner
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Rafael Weingärtner
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rafael Weingärtner
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ionut Biru - https://fleio.com
>>>>>
>>>>
>>>>
>>>> --
>>>> Ionut Biru - https://fleio.com
>>>>
>>>
>>>
>>> --
>>> Rafael Weingärtner
>>>
>>
>>
>> --
>> Ionut Biru - https://fleio.com
>>
>
>
> --
> Rafael Weingärtner
>


-- 
Ionut Biru - https://fleio.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200714/f48a2cc1/attachment-0001.html>


More information about the openstack-discuss mailing list