Open Stack

On 09/24/2012 06:35 AM, Eoghan Glynn wrote:
> 
> 
>> I have been looking at a rework of the current heat-api internal
>> architecture to better support our alarm/metric-collection
>> facilities.
>>
>> There has been some discussion within the team, but I'd like to
>> ensure we
>> have a proper discussion with the whole team and any community
>> members
>> who may be able to contribute ideas, before I proceed with what will
>> be
>> fairly major rework.
>>
>> I know that asalkeld was in discussion with the ceilometer project re
>> merging some of these features into ceilometer, my understanding is
>> that
>> this is unlikely in the short/medium term, hence the proposed changes
>> below, please advise if anyone knows different :)
>>
>> The aim of this rework is as follows:
>> - Provide a better/more-complete Cloudwatch API implementation
>> - Provide an openstack specific ReST API to the Cloudwatch features
>> and
>>   data
>> - Separate ochestration (stack+resource management) from monitoring
>>   (metrics+alarms), aim is to simplify the code, and to provide
>>   better
>>   scalability and performance
>> - Replace the current metadata server, since it combines
>>   resource-metadata and monitoring, and does not authenticate client
>>   connections
>>
>> The proposed changes are illustrated in the following diagram:
>> https://github.com/hardys/diagrams/blob/master/PNG/architecture-overview.png
>>
>> Compare with our current architecture diagram:
>> https://github.com/heat-api/diagrams/blob/master/PNG/architecture-overview.png
>>
>> Summary of proposed changes:
>> - Move resource metadata features of the current metadata-server API
>> into
>>   the new heat ReST API
>> - Modify our cfn-hup implementation to retrieve metadata from the
>> heat
>>   ReST API
>> - Modify our cfn-signal implementation to send WaitCondition metadata
>> to
>>   the heat ReST API, longer term it may be better if the
>>   WaitCondition
>>   implementation is modified to use a Watch rule instead of writing
>>   the
>>   resource metadata.
>> - Move the metric and alarm/watch features of the metadata server
>> into
>>   the Watch API (we already have a Cloudwatch compatible API  which
>>   supports this functionality, medium term perhaps using the proposed
>>   new
>>   Watch ReST API would be better)
>> - Modify our cfn-push-stats implementation to send data to the Watch
>> API
>> - Separate out the metric/watch/alarm heat-engine features into a new
>>   process/daemon ("Watch Engine" in the diagram)
>> - Separate the current watch_rule and watch_data tables from the main
>>   heat DB into a watch-specific separarate DB
>> - Rework the watch DB schema, as it's currently non-ideal, and
>> doesn't
>>   allow us to store event transitions/history which is required for a
>>   complete cloudwatch API implementation
> 
> Hi Steve,
> 
> This proposed separation is much better than deeply embedding
> the CloudWatch API implementation within Heat.
> 
> However, since metrics collection & alarming have much wider
> applicability than their use within orchestration, would it
> make sense to go one step further and hive off the CW/CWA-
> related logic into a completely separate infrastructure that
> could be independently deployed and be developed in parallel
> with the Heat project?
> 

Eoghan,

The end goal of this work is to end with an ideal cloudwatch
architecture.  Making a decision about where cloudwatch lives
permanently limits flexibility.  As a result, I'd rather not limit
flexibility until we have a clear community-driven incentive or decision
to do so.

The main concern I have presently with our cloudwatch implementation is
that cloudwatch cannot be horizontally scaled independently of the heat
engine.  The architecture is fundamentally flawed for this reason.
Steve will resolve this concern with this work.

Regards
-steve

> Cheers,
> Eoghan
> 
> 
>> - Figure out a scheme to put credentials on each instance, so they
>> can
>>   authenticate with the heat/watch APIs
>>
>> The final item is the one I am least clear about, we seem to need the
>> following:
>> - Either per-stack or per-instance credentials, created at stack
>> creation
>>   time
>> - Credentials deployed to the instance in a secure way
>> - Credentials must not expire, or have to be refreshed (e.g via
>> cfn-hup)
>>   in a reliable way.
>> - Note we cannot deploy the initial credentials via cfn-hup since it
>> will
>>   now require credentials (chicken/egg!)
>>
>> Does anyone have any thoughts about the best way to deploy the
>> credentials to each instance?
>>
>> If anyone has any thoughts, suggestions or comments on any of the
>> above
>> (and particularly on the credentials management), they would be
>> appreciated :)
>>
>> Steve
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>