[ceilometer][octavia] polling meters
Rafael Weingärtner
rafaelweingartner at gmail.com
Fri Jul 10 17:24:17 UTC 2020
Sure, this is a minimalistic config I used for testing (watch for the
indentation issues that might happen due to copy/paste into Gmail).
> cat ceilometer/pollsters.d/vpn-connection-dynamic-pollster.yaml
> ---
>
> - name: "dynamic_pollster.network.services.vpn.connection"
> sample_type: "gauge"
> unit: "ipsec_site_connection"
> value_attribute: "status"
> endpoint_type: "network"
> url_path: "v2.0/vpn/ipsec-site-connections"
> metadata_fields:
> - "name"
> - "vpnservice_id"
> - "description"
> - "status"
> - "peer_address"
> value_mapping:
> ACTIVE: "1"
> DOWN: "0"
> metadata_mapping:
> name: "display_name"
> default_value: 0
>
Then, the polling.yaml file
cat ceilometer/polling.yaml | grep -A 3 vpnass
> - name: vpnass_pollsters
> interval: 600
> meters:
> - dynamic_pollster.network.services.vpn.connection
>
And last, but not least, the custom_gnocchi_resources file.
> cat ceilometer/custom_gnocchi_resources.yaml | grep -B 2 -A 9
> "dynamic_pollster.network.services.vpn.connection"
> - resource_type: s2svpn
> metrics:
> dynamic_pollster.network.services.vpn.connection:
> attributes:
> name: resource_metadata.name
> vpnservice_id: resource_metadata.vpnservice_id
> description: resource_metadata.description
> status: resource_metadata.status
> peer_address: resource_metadata.peer_address
> display_name: resource_metadata.display_name
>
Bear in mind that you need to create the Gnocchi resource type.
> gnocchi resource-type show s2svpn
>
> +--------------------------+-----------------------------------------------------------+
> | Field | Value
> |
>
> +--------------------------+-----------------------------------------------------------+
> | attributes/description | max_length=255, min_length=0, required=False,
> type=string |
> | attributes/display_name | max_length=255, min_length=0, required=False,
> type=string |
> | attributes/name | max_length=255, min_length=0, required=False,
> type=string |
> | attributes/peer_address | max_length=255, min_length=0, required=False,
> type=string |
> | attributes/status | max_length=255, min_length=0, required=False,
> type=string |
> | attributes/vpnservice_id | required=False, type=uuid
> |
> | name | s2svpn
> |
> | state | active
> |
>
> +--------------------------+-----------------------------------------------------------+
>
What is the problem you are having?
On Fri, Jul 10, 2020 at 10:50 AM Ionut Biru <ionut at fleio.com> wrote:
> Hi again,
>
> I did not manage to make it work, I cannot figure out how to connect all
> the pieces.
>
> pollsters.d/octavia.yaml https://paste.xinu.at/DERxh1/
> pipeline.yaml https://paste.xinu.at/u1E42/
> polling.yaml https://paste.xinu.at/MZWNs/
> gnocchi_resources.yaml https://paste.xinu.at/j3AX/
> gnocchi_client.py in resources_update_operations
> https://paste.xinu.at/no5/
> gnocchi resource-type show https://paste.xinu.at/7mZIyZ/
> Do you mind if you do a full example
> using "dynamic.network.services.vpn.connection" from
> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html
> ?
>
> Or maybe you can point me to the mistakes made in my configuration?
>
>
> On Tue, Jul 7, 2020 at 2:43 PM Rafael Weingärtner <
> rafaelweingartner at gmail.com> wrote:
>
>> That is the right direction. I don't know why people hard-coded the
>> initial pollsters' configs and did not document the relation between
>> Gnocchi and Ceilometer properly. They (Ceilometer and Gnocchi) are not a
>> single system, but interdependent systems to implement a monitoring
>> solution. Ceilometer is the component that gathers data/information,
>> processes, and then persists it somewhere. Gnocchi is one of the options
>> that Ceilometer can use to persist data. By default, Ceilometer creates
>> some basic configurations in Gnocchi to store data, such as some default
>> resource-types with default attributes. However, we do not need (should
>> not) rely on this default config.
>>
>> You can create and use custom resources to fit the stack to your needs.
>> This can be achieved via `gnocchi resource-type create -a
>> <attribute_name>:<attribute_type>:<required> <name_of_resource_type>` and
>> `gnocchi resource-type create -u
>> <new_attribute_name>:<attribute_type>:<required> <name_of_resource_type>`.
>> Then, in the `custom_gnocchi_resources.yaml` (if you use Kolla-ansible),
>> you can customize the mapping of metrics to resource-types in Gnocchi.
>>
>> On Tue, Jul 7, 2020 at 7:49 AM Ionut Biru <ionut at fleio.com> wrote:
>>
>>> Hello again,
>>>
>>> What's the proper way to handle dynamic pollsters in gnocchi ?
>>> Right now ceilometer returns:
>>>
>>> WARNING ceilometer.publisher.gnocchi [-] metric dynamic.network.octavia
>>> is not handled by Gnocchi
>>>
>>> I found
>>> https://docs.openstack.org/ceilometer/latest/contributor/new_resource_types.html
>>> but I'm not sure if is the right direction.
>>>
>>> On Tue, Jul 7, 2020 at 10:52 AM Ionut Biru <ionut at fleio.com> wrote:
>>>
>>>> Seems to work fine now. Thanks.
>>>>
>>>> On Mon, Jul 6, 2020 at 8:12 PM Rafael Weingärtner <
>>>> rafaelweingartner at gmail.com> wrote:
>>>>
>>>>> It looks like a coding error that we left behind during a major
>>>>> refactoring that we introduced upstream.
>>>>> I created a patch for it. Can you check/review and test it?
>>>>> https://review.opendev.org/739555
>>>>>
>>>>> On Mon, Jul 6, 2020 at 11:17 AM Ionut Biru <ionut at fleio.com> wrote:
>>>>>
>>>>>> Hi Rafael,
>>>>>>
>>>>>> I have an error and I cannot resolve it myself.
>>>>>>
>>>>>> https://paste.xinu.at/LEfdXD/
>>>>>>
>>>>>> Do you happen to know what's wrong?
>>>>>>
>>>>>> endpoint list https://paste.xinu.at/v3j1jl/
>>>>>> octavia.yaml https://paste.xinu.at/TIxfOz/
>>>>>> polling.yaml https://paste.xinu.at/oBEFj/
>>>>>> pipeline.yaml https://paste.xinu.at/qvEdTX/
>>>>>>
>>>>>>
>>>>>> On Sat, Jul 4, 2020 at 1:10 AM Rafael Weingärtner <
>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>
>>>>>>> Good catch. I fixed the docs.
>>>>>>> https://review.opendev.org/#/c/739288/
>>>>>>>
>>>>>>> On Fri, Jul 3, 2020 at 1:59 PM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I just noticed that the example
>>>>>>>> dynamic.network.services.vpn.connection from
>>>>>>>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html has
>>>>>>>> the wrong indentation.
>>>>>>>> This https://paste.xinu.at/6PTfsM/ is loaded without any error.
>>>>>>>>
>>>>>>>> Now I have to see why is not polling from it
>>>>>>>>
>>>>>>>> On Fri, Jul 3, 2020 at 7:19 PM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Rafael,
>>>>>>>>>
>>>>>>>>> I think I applied all the reviews successfully but I tried to do
>>>>>>>>> an octavia dynamic poller but I have couples of errors.
>>>>>>>>>
>>>>>>>>> Here is the octavia.yaml: https://paste.xinu.at/kDN6SV/
>>>>>>>>> Error is about syntax error near name:
>>>>>>>>> https://paste.xinu.at/MHgDBY/
>>>>>>>>>
>>>>>>>>> if i remove the - in front of name like this:
>>>>>>>>> https://paste.xinu.at/K7s5I8/
>>>>>>>>> The error is different this time: https://paste.xinu.at/zWdC0U/
>>>>>>>>>
>>>>>>>>> Is there something I missed or is something wrong in yaml?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jul 2, 2020 at 5:50 PM Rafael Weingärtner <
>>>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Since the merging window for ussuri was long passed for those
>>>>>>>>>>> commits, is it safe to assume that it will not land in stable/ussuri at all
>>>>>>>>>>> and those will be available for victoria?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I would say so. We are lacking people to review and then merge it.
>>>>>>>>>>
>>>>>>>>>> How safe is to cherry pick those commits and use them in
>>>>>>>>>>> production?
>>>>>>>>>>>
>>>>>>>>>> As long as the person executing the cherry-picks, and maintaining
>>>>>>>>>> the code knows what she/he is doing, you should be safe. The guys that are
>>>>>>>>>> using this implementation (and others that I and my colleagues proposed),
>>>>>>>>>> have a few openstack components that are customized with the
>>>>>>>>>> patches/enhancements/extensions we developed so far; this means, they are
>>>>>>>>>> not using the community version, but something in-between (the community
>>>>>>>>>> releases + the patches we did). Of course, it is only possible, because we
>>>>>>>>>> are the ones creating and maintaining these codes; therefore, we can assure
>>>>>>>>>> quality for production.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 2, 2020 at 9:43 AM Ionut Biru <ionut at fleio.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Rafael,
>>>>>>>>>>>
>>>>>>>>>>> Since the merging window for ussuri was long passed for those
>>>>>>>>>>> commits, is it safe to assume that it will not land in stable/ussuri at all
>>>>>>>>>>> and those will be available for victoria?
>>>>>>>>>>>
>>>>>>>>>>> How safe is to cherry pick those commits and use them in
>>>>>>>>>>> production?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Apr 24, 2020 at 3:06 PM Rafael Weingärtner <
>>>>>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> The dynamic pollster in Ceilometer will be first released in
>>>>>>>>>>>> Ussuri. However, there are some important PRs still waiting for a merge,
>>>>>>>>>>>> that might be important for your use case:
>>>>>>>>>>>> * https://review.opendev.org/#/c/722092/
>>>>>>>>>>>> * https://review.opendev.org/#/c/715180/
>>>>>>>>>>>> * https://review.opendev.org/#/c/715289/
>>>>>>>>>>>> * https://review.opendev.org/#/c/679999/
>>>>>>>>>>>> * https://review.opendev.org/#/c/709807/
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Apr 24, 2020 at 8:18 AM Carlos Goncalves <
>>>>>>>>>>>> cgoncalves at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 12:20 PM Ionut Biru <ionut at fleio.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I want to meter the loadbalancer into gnocchi for billing
>>>>>>>>>>>>>> purposes in stein/train and ceilometer doesn't support dynamic pollsters.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think I misunderstood your use case, sorry. I read it as if
>>>>>>>>>>>>> you wanted to know "if a loadbalancer was deployed and has status active".
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Until I upgrade to Ussuri, is there a way to accomplish this?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not sure Ceilometer supports it even in Ussuri. I'll defer
>>>>>>>>>>>>> to the Ceilometer project.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 12:45 PM Carlos Goncalves <
>>>>>>>>>>>>>> cgoncalves at redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Ionut,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 11:27 AM Ionut Biru <ionut at fleio.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello guys,
>>>>>>>>>>>>>>>> I was trying to add in polling.yaml and pipeline from
>>>>>>>>>>>>>>>> ceilometer the following:
>>>>>>>>>>>>>>>> - network.services.lb.active.connections
>>>>>>>>>>>>>>>> - network.services.lb.health_monitor
>>>>>>>>>>>>>>>> - network.services.lb.incoming.bytes
>>>>>>>>>>>>>>>> - network.services.lb.listener
>>>>>>>>>>>>>>>> - network.services.lb.loadbalancer
>>>>>>>>>>>>>>>> - network.services.lb.member
>>>>>>>>>>>>>>>> - network.services.lb.outgoing.bytes
>>>>>>>>>>>>>>>> - network.services.lb.pool
>>>>>>>>>>>>>>>> - network.services.lb.total.connections
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But it doesn't work, I think they are for the old lbs that
>>>>>>>>>>>>>>>> were supported in neutron.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I found
>>>>>>>>>>>>>>>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html
>>>>>>>>>>>>>>>> but this is not available in stein or train.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was wondering if there is a way to meter
>>>>>>>>>>>>>>>> loadbalancers from octavia.
>>>>>>>>>>>>>>>> I mostly want for start to just meter if a loadbalancer was
>>>>>>>>>>>>>>>> deployed and has status active.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You can get the provisioning and operating status of Octavia
>>>>>>>>>>>>>>> load balancers via the Octavia API. There is also an API endpoint that
>>>>>>>>>>>>>>> returns the full load balancer status tree [1].
>>>>>>>>>>>>>>> Additionally, Octavia has three API endpoints for
>>>>>>>>>>>>>>> statistics [2][3][4].
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I hope this helps with your use case.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Carlos
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-the-load-balancer-status-tree-detail#get-the-load-balancer-status-tree
>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-load-balancer-statistics-detail#get-load-balancer-statistics
>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-listener-statistics-detail#get-listener-statistics
>>>>>>>>>>>>>>> [4]
>>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=show-amphora-statistics-detail#show-amphora-statistics
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Rafael Weingärtner
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Rafael Weingärtner
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rafael Weingärtner
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ionut Biru - https://fleio.com
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rafael Weingärtner
>>>>>
>>>>
>>>>
>>>> --
>>>> Ionut Biru - https://fleio.com
>>>>
>>>
>>>
>>> --
>>> Ionut Biru - https://fleio.com
>>>
>>
>>
>> --
>> Rafael Weingärtner
>>
>
>
> --
> Ionut Biru - https://fleio.com
>
--
Rafael Weingärtner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200710/ea039070/attachment-0001.html>
More information about the openstack-discuss
mailing list