Open Stack

On 02/08/13 13:26 -0300, Sandy Walsh wrote:
>
>
>On 08/02/2013 12:27 PM, Eoghan Glynn wrote:
>>
>>> On 08/01/2013 07:22 PM, Doug Hellmann wrote:
>>>>
>>>>
>>>>
>>>> On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh <sandy.walsh at rackspace.com
>>>> <mailto:sandy.walsh at rackspace.com>> wrote:
>>>>
>>>>     Hey y'all,
>>>>
>>>>     I've had a little thorn in my claw on this topic for a while and
>>>>     thought
>>>>     I'd ask the larger group.
>>>>
>>>>     I applaud the efforts of the people working on the alarming additions
>>>>     to
>>>>     Ceilometer, but I've got concerns that we're packaging things the
>>>>     wrong way.
>>>>
>>>>     I fear we're making another Zenoss/Nagios with Ceilometer. It's trying
>>>>     to do too much.
>>>>
>>>>     The current trend in the monitoring work (#monitoringlove) is to build
>>>>     your own stack from a series of components. These components take in
>>>>     inputs, process them and spit out outputs.
>>>>     Collectors/Transformers/Publishers. This is the model CM is built on.
>>>>
>>>>     Making an all-singing-all-dancing monolithic monitoring package is the
>>>>     old way of building these tools. People want to use best-of-breed
>>>>     components for their monitoring stack. I'd like to be able to use
>>>>     reimann.io <http://reimann.io> for my stream manager, diamond for my
>>>>     collector, logstash for
>>>>     my parser, etc. Alarming should just be another consumer.
>>>>
>>>>     CM should do one thing well. Collect data from openstack, store and
>>>>     process them, and make them available to other systems via the API or
>>>>     publishers. That's all. It should understand these events better than
>>>>     any other product out there. It should be able to produce meaningful
>>>>     metrics/meters from these events.
>>>>
>>>>     Alarming should be yet another consumer of the data CM produces. Done
>>>>     right, the If-This-Then-That nature of the alarming tool could be
>>>>     re-used by the orchestration team or perhaps even scheduling.
>>>>     Intertwining it with CM is making the whole thing too complex and rigid
>>>>     (imho).

Heat is using the alarm api.

>>>>
>>>>     CM should be focused on extending our publishers and input plug-ins.
>>>>
>>>>     I'd like to propose that alarming becomes its own project outside of
>>>>     Ceilometer. Or, at the very least, its own package, external of the
>>>>     Ceilometer code base. Perhaps it still lives under the CM moniker, but
>>>>     packaging-wise, I think it should live as a separate code base.
>>>>
>>>> It is currently implemented as a pair of daemons (one to monitor the
>>>> alarm state, another to send the notifications). Both daemons use a
>>>> ceilometer client to talk to the REST API to consume the sample data or
>>>> get the alarm details, as required. It looks like alarms are triggered
>>>> by sending RPC cast message, and that those in turn trigger the webhook
>>>> invocation. That seems pretty loosely coupled, as far as the runtime
>>>> goes. Granted, it's still in the main ceilometer code tree, but that
>>>> doesn't say anything about how the distros will package it.
>>>>
>>>> I'll admit I haven't been closely involved in the development of this
>>>> feature, so maybe my quick review of the code missed something that is
>>>> bringing on this sentiment?
>>>
>>> No, you hit the nail on the head. It's nothing with the implementation,
>>> it's purely with the packaging and having it co-exist within ceilometer.
>>> Since it has its own services, uses Oslo, the CM client and operates via
>>> the public API, it should be able to live outside the main CM codebase.
>>> My concern is that it has a different mandate than CM (or the CM mandate
>>> is too broad).
>>>
>>> What really brought it on for me was doing code reviews for CM and
>>> hitting all this alarm stuff and thinking "this is mental context switch
>>> from what CM does, it really doesn't belong here." (though I'm happy to
>>> help out with the reviews)
>>>
>>> -S
>>
>> Hi Sandy,
>>
>> In terms of distro packaging, the case that I'm most familiar (Fedora & derivatives)
>> already splits out the ceilometer packaging in a fairly fine-grained manner (with
>> separate RPMs for the various services and agents). I'd envisage a similar packaging
>> approach will be followed for the alarming services, so for deployments for which
>> alarming is not required, this functionality won't be foisted on anyone.
>
>Thanks for the feedback Eoghan.
>
>I don't imagine that should be a big problem. Packaging in the sense of
>the code base is different issue. If, for all intents and purposes,
>alarming is a separate system: uses external api's, only uses sanctioned
>CM client libraries, is distro packaged separately and optionally
>installed/deployed then I don't understand why it has to live in the CM
>codebase?
>
>> Now we could think about splitting it out even further to aid the sort of composability
>> you desire, however this functionality is needed by Heat, so it makes sense for it to
>> live in one of the integrated projects (as opposed to a newly incubated split-off).
>
>I'm not purposing it becomes a newly incubated project. I think it makes
>sense to live under the CM moniker. But code-wise, it should be a
>separate repo. This wouldn't pose any problem since -infra already does
>this with its many repos.
>
>> In terms of the context switching required for reviewing alarming patches, I don't
>> see that as a huge issue for two reasons:
>>
>>  - larger projects such as nova already require immensely greater context switching
>>    of it's team of core devs
>>
>>  - core devs can and do concentrate their reviewing on some functional sub-domains,
>>    i.e. everyone doesn't have to the entire spectrum of patches
>
>Having it in a separate repo would lessen the chance of leakage between
>dependencies. Are there libraries that alarming needs that CM doesn't
>need? Is the db model growing in ways that aren't core to CM? Is there

No, alarming has no new depenancies.

>greater volatility in the project overall because of the increased
>number of branches going in? Will new developers have a harder time
>onboarding due to more code?
>
>I think the answer is yes across the board.

No. It is called community building. The more people you have involved in
a project the more you can get things done (more bugs fixed etc...). 
It's about people working together.

>
>We're all in agreement that alarming is essentially standalone already
er, no. There is a rest api and a db api that are built in.

>... now it just needs its own repo.

Huge -1 to this anti-effort.

We have to move alarming out because you don't like looking at it?
Dam, this sounds really arrogant.


-Angus

>> That being said, the efforts of team members such as yourself who make an obvious
>> effort to review a wide range of patches, is of course appreciated!
>> Cheers,
>> Eoghan
>>
>>>
>>>>
>>>> Doug
>>>>
>>>>
>>>>
>>>>     Please, change my view. :)
>>>>
>>>>     -S
>>>>
>>>>     Side note: I might even argue that the statistical features of CM are
>>>>     going a little too far. My only counter-argument is that statistics are
>>>>     the only way to prevent us from sending large amounts of data over the
>>>>     wire for post-processing. But that's a separate discussion.
>>>>
>>>>
>>>>     _______________________________________________
>>>>     OpenStack-dev mailing list
>>>>     OpenStack-dev at lists.openstack.org
>>>>     <mailto:OpenStack-dev at lists.openstack.org>
>>>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev