Open Stack

> I have been looking at a rework of the current heat-api internal
> architecture to better support our alarm/metric-collection
> facilities.
> 
> There has been some discussion within the team, but I'd like to
> ensure we
> have a proper discussion with the whole team and any community
> members
> who may be able to contribute ideas, before I proceed with what will
> be
> fairly major rework.
> 
> I know that asalkeld was in discussion with the ceilometer project re
> merging some of these features into ceilometer, my understanding is
> that
> this is unlikely in the short/medium term, hence the proposed changes
> below, please advise if anyone knows different :)
> 
> The aim of this rework is as follows:
> - Provide a better/more-complete Cloudwatch API implementation
> - Provide an openstack specific ReST API to the Cloudwatch features
> and
>   data
> - Separate ochestration (stack+resource management) from monitoring
>   (metrics+alarms), aim is to simplify the code, and to provide
>   better
>   scalability and performance
> - Replace the current metadata server, since it combines
>   resource-metadata and monitoring, and does not authenticate client
>   connections
> 
> The proposed changes are illustrated in the following diagram:
> https://github.com/hardys/diagrams/blob/master/PNG/architecture-overview.png
> 
> Compare with our current architecture diagram:
> https://github.com/heat-api/diagrams/blob/master/PNG/architecture-overview.png
> 
> Summary of proposed changes:
> - Move resource metadata features of the current metadata-server API
> into
>   the new heat ReST API
> - Modify our cfn-hup implementation to retrieve metadata from the
> heat
>   ReST API
> - Modify our cfn-signal implementation to send WaitCondition metadata
> to
>   the heat ReST API, longer term it may be better if the
>   WaitCondition
>   implementation is modified to use a Watch rule instead of writing
>   the
>   resource metadata.
> - Move the metric and alarm/watch features of the metadata server
> into
>   the Watch API (we already have a Cloudwatch compatible API  which
>   supports this functionality, medium term perhaps using the proposed
>   new
>   Watch ReST API would be better)
> - Modify our cfn-push-stats implementation to send data to the Watch
> API
> - Separate out the metric/watch/alarm heat-engine features into a new
>   process/daemon ("Watch Engine" in the diagram)
> - Separate the current watch_rule and watch_data tables from the main
>   heat DB into a watch-specific separarate DB
> - Rework the watch DB schema, as it's currently non-ideal, and
> doesn't
>   allow us to store event transitions/history which is required for a
>   complete cloudwatch API implementation

Hi Steve,

This proposed separation is much better than deeply embedding
the CloudWatch API implementation within Heat.

However, since metrics collection & alarming have much wider
applicability than their use within orchestration, would it
make sense to go one step further and hive off the CW/CWA-
related logic into a completely separate infrastructure that
could be independently deployed and be developed in parallel
with the Heat project?

Cheers,
Eoghan


> - Figure out a scheme to put credentials on each instance, so they
> can
>   authenticate with the heat/watch APIs
> 
> The final item is the one I am least clear about, we seem to need the
> following:
> - Either per-stack or per-instance credentials, created at stack
> creation
>   time
> - Credentials deployed to the instance in a secure way
> - Credentials must not expire, or have to be refreshed (e.g via
> cfn-hup)
>   in a reliable way.
> - Note we cannot deploy the initial credentials via cfn-hup since it
> will
>   now require credentials (chicken/egg!)
> 
> Does anyone have any thoughts about the best way to deploy the
> credentials to each instance?
> 
> If anyone has any thoughts, suggestions or comments on any of the
> above
> (and particularly on the credentials management), they would be
> appreciated :)
> 
> Steve
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>