Open Stack

Hello,

as you may know, Paunch [0] was originally developed for simple 
containers management in TripleO. Today we have Paunch deprecated for 
the preference of ansible module [1]. It uses ansible async executions 
and performs management actions in batches.

While that's great and novel and works faster, Paunch's logs were simple 
to follow and its code was easy to debug with pdb breakpoints. Howewer, 
debugging experience and overall observability (please excuse me that 
buzzword) has changed [2], when switched it to ansible.

In order improve that situation, I propose that data aggregator [3] for 
post-processing of the containers data under the module's management. It 
is based on setting facts, using already developed custom filters [4], 
and ansible's set_stats [5]. That data is just a (big) dictionary that 
contains:
* input containers configs fetched from json files for all deploy steps,
* snapshotted runtime info as it was consumed by the containers 
management tasks,
* internal processing results (identified logical state, like 'changed' 
or 'broken')

That is for all of the batches of managed containers in a deployment, 
per hosts basis. And that's basically it. How to use that data is out of 
scope at this point of discussion.

Please share your thoughts on the topic. Note that the aggregator 
itiself is simple (~140 LOC). Please don't call it over-engineered :)
It *only* allows users and tripleo devs to build whatever complex and 
over-engineered data analysing pipelines on top of it. Those may be 
colored graphs of dependencies and states, or simple json parsing, like 
that prototype [5] (WIP, only for a standalone deployments).

Wrapping it up, it's up to devs and users how to use that data 
aggregator. Let's just simplify their lives with debugging containers 
management framework please.

[0] https://opendev.org/openstack/paunch

[1] 
https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_container_manage

[2] https://launchpad.net/bugs/1879455

[3] https://review.opendev.org/#/q/topic:bug/1879455

[4] 
https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/ansible_plugins/filter/helpers.py

[5] https://docs.ansible.com/ansible/latest/modules/set_stats_module.html

-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando