Open Stack

hi folks,

i want to start off by thanking everyone for joining the telemetry 
related discussions at the Tokyo summit -- we got some great feedback 
and ideas. similar to the last summit, here's a rundown of items that we 
talked about[1] and will be tracking in the upcoming cycle. as before, 
this is a (chaotic) brain dump and does not necessarily reflect any 
prioritisation.

note: the following is split into different sections, each representing 
a service provided under the telemetry umbrella. these projects are 
discretely managed but with tight collaboration.


-- Aodh (alarming service) --

- client - since we split alarming functionality from Ceilometer into 
it's own unique service[2], aodhclient support will be added so 
ceilometerclient functionality does not become overloaded with unrelated 
alarming code.
- user interface - to improve usability, support for CRUD operations of 
alarms will be added to horizon [3]
- horizontal scaling - the existing event alarm support added in 
Liberty[4] handles a single evaluator. multiple worker support will be 
added to enable better scaling
- simplifying combination alarm - combination alarms allowed flexibility 
of reusing threshold alarms but limited the use case to AND conditions 
and added evaluation ordering complexity. these drawbacks will be 
addressed by a new composite alarm [5]
- testing - tempest and grenade plugin testing support to be added in 
addition to existing unit/functional tests


-- Ceilometer (data collection service) --

- example reference architecture - to improve the consumption of 
Ceilometer, performance study will be done to build reference 
architecture. additional example configurations will be added to enable 
easier entry based on use case.
- housekeeping - alarming and rpc functionality were deprecated in 
Kilo/Liberty[6]. to ensure a tidy code base, it's time to clean house or 
as some devs like to put it: burn it down.[*]
- rolling upgrades - document the process of upgrading
- refined polling - the polling agent(s) now exclusively poll data and 
defer processing to notification agent(s). because of this, we can 
create a more tailored polling and pipeline configuration experience.
- improved polling distribution - currently, the cache is shared within 
a process. to better enable task distribution, we should enable sharing 
the cache across processes.
- polling metadata caching - we improved the caching mechanism in 
Liberty to minimise the load caused by Ceilometer polling. further 
improvements can be made to minimise the number of calls in general.[7]
- resource caching - to improve write speed, a cache will be implemented 
in the collector to minimise unnecessary writes[8]
- batch db writing - to minimise writes, batched writing of data will be 
added to collector[9]
- componentisation part 2 - Ceilometer handles meters[10] and 
events[11]. we need to make it pluggable to offer better flexibility.
- testing - tempest and grenade plugin testing support to be added in 
addition to existing unit/functional tests. additionally, multi-node 
testing to test upgrade path.


-- Gnocchi (time-series database and resource indexing service) --

- metric archive sharding - to improve performance of very large data 
sets (ie. second-by-second granularity), we can split the archive when 
updating data to avoid transferring entire archive set.
- dynamic resource creation - currently to create a new resource type, a 
new model and migration needs to be added. we need to make this creation 
dynamic to allow for more resource types.
- proliferate gnocchiclient adoption - gnocchiclient is now available. 
to ensure consistent usage, it should be adopted in Ceilometer and Aodh.
- testing - tempest and grenade plugin testing support to be added in 
addition to existing unit/functional tests


you can sign up for work items on the etherpad[12]. as always, you're 
more than welcome to propose addition ideas on irc:#openstack-ceilometer 
(to be #openstack-telemetry) and to openstack-dev using  
[telemetry]/[ceilometer]/[aodh]/[gnocchi] in subject.

as always we will continue to work with externally managed projects[13].

[1] 
https://wiki.openstack.org/wiki/Design_Summit/Mitaka/Etherpads#Ceilometer
[2] 
http://lists.openstack.org/pipermail/openstack-dev/2015-September/073897.html
[3] 
https://blueprints.launchpad.net/openstack-ux/+spec/horizon-alarm-management
[4] 
http://specs.openstack.org/openstack/ceilometer-specs/specs/liberty/event-alarm-evaluator.html
[5] https://review.openstack.org/#/c/208786/
[6] 
https://wiki.openstack.org/wiki/ReleaseNotes/Liberty#OpenStack_Telemetry_.28Ceilometer.29
[7] https://review.openstack.org/#/c/209799/
[8] https://review.openstack.org/#/c/203109/
[9] https://review.openstack.org/#/c/234831/
[10] http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html
[11] http://docs.openstack.org/admin-guide-cloud/telemetry-events.html
[12] https://etherpad.openstack.org/p/mitaka-telemetry-todos
[13] https://wiki.openstack.org/wiki/Ceilometer#Externally_Managed
[*] cdent

** turns off management mode **

cheers,

-- 
gord