Open Stack

Hi again,

I did not manage to make it work, I cannot figure out how to connect all
the pieces.

pollsters.d/octavia.yaml  https://paste.xinu.at/DERxh1/
pipeline.yaml https://paste.xinu.at/u1E42/
polling.yaml https://paste.xinu.at/MZWNs/
gnocchi_resources.yaml https://paste.xinu.at/j3AX/
gnocchi_client.py in resources_update_operations  https://paste.xinu.at/no5/

gnocchi resource-type show https://paste.xinu.at/7mZIyZ/
Do you mind if you do a full example
using  "dynamic.network.services.vpn.connection" from
https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html
 ?

Or maybe you can point me to the mistakes made in my configuration?


On Tue, Jul 7, 2020 at 2:43 PM Rafael Weingärtner <
rafaelweingartner at gmail.com> wrote:

> That is the right direction. I don't know why people hard-coded the
> initial pollsters' configs and did not document the relation between
> Gnocchi and Ceilometer properly. They (Ceilometer and Gnocchi) are not a
> single system, but interdependent systems to implement a monitoring
> solution. Ceilometer is the component that gathers data/information,
> processes, and then persists it somewhere. Gnocchi is one of the options
> that Ceilometer can use to persist data. By default, Ceilometer creates
> some basic configurations in Gnocchi to store data, such as some default
> resource-types with default attributes. However, we do not need (should
> not) rely on this default config.
>
> You can create and use custom resources to fit the stack to your needs.
> This can be achieved via `gnocchi resource-type create -a
> <attribute_name>:<attribute_type>:<required> <name_of_resource_type>` and
> `gnocchi resource-type create -u
> <new_attribute_name>:<attribute_type>:<required> <name_of_resource_type>`.
> Then, in the `custom_gnocchi_resources.yaml` (if you use Kolla-ansible),
> you can customize the mapping of metrics to resource-types in Gnocchi.
>
> On Tue, Jul 7, 2020 at 7:49 AM Ionut Biru <ionut at fleio.com> wrote:
>
>> Hello again,
>>
>> What's the proper way to handle dynamic pollsters in gnocchi ?
>> Right now ceilometer returns:
>>
>> WARNING ceilometer.publisher.gnocchi [-] metric dynamic.network.octavia
>> is not handled by Gnocchi
>>
>> I found
>> https://docs.openstack.org/ceilometer/latest/contributor/new_resource_types.html
>> but I'm not sure if is the right direction.
>>
>> On Tue, Jul 7, 2020 at 10:52 AM Ionut Biru <ionut at fleio.com> wrote:
>>
>>> Seems to work fine now. Thanks.
>>>
>>> On Mon, Jul 6, 2020 at 8:12 PM Rafael Weingärtner <
>>> rafaelweingartner at gmail.com> wrote:
>>>
>>>> It looks like a coding error that we left behind during a major
>>>> refactoring that we introduced upstream.
>>>> I created a patch for it. Can you check/review and test it?
>>>> https://review.opendev.org/739555
>>>>
>>>> On Mon, Jul 6, 2020 at 11:17 AM Ionut Biru <ionut at fleio.com> wrote:
>>>>
>>>>> Hi Rafael,
>>>>>
>>>>> I have an error and I cannot resolve it myself.
>>>>>
>>>>> https://paste.xinu.at/LEfdXD/
>>>>>
>>>>> Do you happen to know what's wrong?
>>>>>
>>>>> endpoint list https://paste.xinu.at/v3j1jl/
>>>>> octavia.yaml https://paste.xinu.at/TIxfOz/
>>>>> polling.yaml https://paste.xinu.at/oBEFj/
>>>>> pipeline.yaml https://paste.xinu.at/qvEdTX/
>>>>>
>>>>>
>>>>> On Sat, Jul 4, 2020 at 1:10 AM Rafael Weingärtner <
>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>
>>>>>> Good catch. I fixed the docs.
>>>>>> https://review.opendev.org/#/c/739288/
>>>>>>
>>>>>> On Fri, Jul 3, 2020 at 1:59 PM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I just noticed that the example
>>>>>>> dynamic.network.services.vpn.connection from
>>>>>>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html has
>>>>>>> the wrong indentation.
>>>>>>> This https://paste.xinu.at/6PTfsM/ is loaded without any error.
>>>>>>>
>>>>>>> Now I have to see why is not polling from it
>>>>>>>
>>>>>>> On Fri, Jul 3, 2020 at 7:19 PM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>>
>>>>>>>> Hi Rafael,
>>>>>>>>
>>>>>>>> I think I applied all the reviews successfully but I tried to do an
>>>>>>>> octavia dynamic poller but I have couples of errors.
>>>>>>>>
>>>>>>>> Here is the octavia.yaml: https://paste.xinu.at/kDN6SV/
>>>>>>>> Error is about syntax error near name:
>>>>>>>> https://paste.xinu.at/MHgDBY/
>>>>>>>>
>>>>>>>> if i remove the - in front of name like this:
>>>>>>>> https://paste.xinu.at/K7s5I8/
>>>>>>>> The error is different this time: https://paste.xinu.at/zWdC0U/
>>>>>>>>
>>>>>>>> Is there something I missed or is something wrong  in yaml?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jul 2, 2020 at 5:50 PM Rafael Weingärtner <
>>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Since the merging window for ussuri was long passed for those
>>>>>>>>>> commits, is it safe to assume that it will not land in stable/ussuri at all
>>>>>>>>>> and those will be available for victoria?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I would say so. We are lacking people to review and then merge it.
>>>>>>>>>
>>>>>>>>> How safe is to cherry pick those commits and use them in
>>>>>>>>>> production?
>>>>>>>>>>
>>>>>>>>> As long as the person executing the cherry-picks, and maintaining
>>>>>>>>> the code knows what she/he is doing, you should be safe. The guys that are
>>>>>>>>> using this implementation (and others that I and my colleagues proposed),
>>>>>>>>> have a few openstack components that are customized with the
>>>>>>>>> patches/enhancements/extensions we developed so far; this means, they are
>>>>>>>>> not using the community version, but something in-between (the community
>>>>>>>>> releases + the patches we did). Of course, it is only possible, because we
>>>>>>>>> are the ones creating and maintaining these codes; therefore, we can assure
>>>>>>>>> quality for production.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jul 2, 2020 at 9:43 AM Ionut Biru <ionut at fleio.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Rafael,
>>>>>>>>>>
>>>>>>>>>> Since the merging window for ussuri was long passed for those
>>>>>>>>>> commits, is it safe to assume that it will not land in stable/ussuri at all
>>>>>>>>>> and those will be available for victoria?
>>>>>>>>>>
>>>>>>>>>> How safe is to cherry pick those commits and use them in
>>>>>>>>>> production?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 24, 2020 at 3:06 PM Rafael Weingärtner <
>>>>>>>>>> rafaelweingartner at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> The dynamic pollster in Ceilometer will be first released in
>>>>>>>>>>> Ussuri. However, there are some important PRs still waiting for a merge,
>>>>>>>>>>> that might be important for your use case:
>>>>>>>>>>> * https://review.opendev.org/#/c/722092/
>>>>>>>>>>> * https://review.opendev.org/#/c/715180/
>>>>>>>>>>> * https://review.opendev.org/#/c/715289/
>>>>>>>>>>> * https://review.opendev.org/#/c/679999/
>>>>>>>>>>> * https://review.opendev.org/#/c/709807/
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Apr 24, 2020 at 8:18 AM Carlos Goncalves <
>>>>>>>>>>> cgoncalves at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Apr 24, 2020 at 12:20 PM Ionut Biru <ionut at fleio.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I want to meter the loadbalancer into gnocchi for billing
>>>>>>>>>>>>> purposes in stein/train and ceilometer doesn't support dynamic pollsters.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I think I misunderstood your use case, sorry. I read it as if
>>>>>>>>>>>> you wanted to know "if a loadbalancer was deployed and has status active".
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Until I upgrade to Ussuri, is there a way to accomplish this?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure Ceilometer supports it even in Ussuri. I'll defer
>>>>>>>>>>>> to the Ceilometer project.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 12:45 PM Carlos Goncalves <
>>>>>>>>>>>>> cgoncalves at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Ionut,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Apr 24, 2020 at 11:27 AM Ionut Biru <ionut at fleio.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello guys,
>>>>>>>>>>>>>>> I was trying to add in polling.yaml and pipeline from
>>>>>>>>>>>>>>> ceilometer the following:
>>>>>>>>>>>>>>>    - network.services.lb.active.connections
>>>>>>>>>>>>>>>       - network.services.lb.health_monitor
>>>>>>>>>>>>>>>       - network.services.lb.incoming.bytes
>>>>>>>>>>>>>>>       - network.services.lb.listener
>>>>>>>>>>>>>>>       - network.services.lb.loadbalancer
>>>>>>>>>>>>>>>       - network.services.lb.member
>>>>>>>>>>>>>>>       - network.services.lb.outgoing.bytes
>>>>>>>>>>>>>>>       - network.services.lb.pool
>>>>>>>>>>>>>>>       - network.services.lb.total.connections
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But it doesn't work, I think they are for the old lbs that
>>>>>>>>>>>>>>> were supported in neutron.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I found
>>>>>>>>>>>>>>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html
>>>>>>>>>>>>>>> but this is not available in stein or train.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I was wondering if there is a way to meter
>>>>>>>>>>>>>>> loadbalancers from octavia.
>>>>>>>>>>>>>>> I mostly want for start to just meter if a loadbalancer was
>>>>>>>>>>>>>>> deployed and has status active.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You can get the provisioning and operating status of Octavia
>>>>>>>>>>>>>> load balancers via the Octavia API. There is also an API endpoint that
>>>>>>>>>>>>>> returns the full load balancer status tree [1].
>>>>>>>>>>>>>> Additionally, Octavia has three API endpoints for statistics
>>>>>>>>>>>>>> [2][3][4].
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I hope this helps with your use case.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>> Carlos
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-the-load-balancer-status-tree-detail#get-the-load-balancer-status-tree
>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-load-balancer-statistics-detail#get-load-balancer-statistics
>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=get-listener-statistics-detail#get-listener-statistics
>>>>>>>>>>>>>> [4]
>>>>>>>>>>>>>> https://docs.openstack.org/api-ref/load-balancer/v2/?expanded=show-amphora-statistics-detail#show-amphora-statistics
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Rafael Weingärtner
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Rafael Weingärtner
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Ionut Biru - https://fleio.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rafael Weingärtner
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ionut Biru - https://fleio.com
>>>>>
>>>>
>>>>
>>>> --
>>>> Rafael Weingärtner
>>>>
>>>
>>>
>>> --
>>> Ionut Biru - https://fleio.com
>>>
>>
>>
>> --
>> Ionut Biru - https://fleio.com
>>
>
>
> --
> Rafael Weingärtner
>


-- 
Ionut Biru - https://fleio.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200710/d19cea30/attachment-0001.html>