[openstack-dev] [Openstack] [Ceilometer][Architecture] Transformers in Kilo vs Liberty(and Mitaka)
gord at live.ca
Thu Apr 14 12:00:18 UTC 2016
On 14/04/2016 5:28 AM, Nadya Shakhat wrote:
> Hi Gordon,
> I'd like to add some clarifications and comments.
> this is not entirely accurate pre-polling change, the polling agents
> publish one message per sample. not the polling agents publish one
> message per interval (multiple samples).
> Looks like there is some misunderstanding here. In the code, there is
> "batch_polled_samples" option. You can switch it off and get the result
> you described, but it's True by default. See
right... the polling agents are by default to publish one message per
interval as i said (if you s/not/now/) where as before it was publishing
1 message per sample. i don't see why that's a bad thing?
> You wrote:
> the polling change is not related to coordination work in notification.
> the coordination work was to handle HA / multiple notification agents.
> regardless polling change, this must exist.
> transformers are already optional. they can be removed from
> pipeline.yaml if not required (and thus coordination can be disabled).
> So, coordination is needed only to support transformations. Polling
> change does relate to this because it has brought additional
> transformations on notification agent side. I suggest to pay attention
> to the existing use cases. In real life, people use transformers for
> polling-based metrics only. The most important use case for
> transformation is Heat autoscaling. It usually based on cpu_util. Before
> Liberty, we were able not to use coordination for notification agent to
> support the autoscaling usecase. In Liberty we cannot support it without
> Redis. Now "transformers are already optional", that's true. But I think
> it's better to add some restrictions like "we don't support
> transformations for notifications" and have transformers optional on
> polling-agent only instead of introducing such a comprehensive
i'm not sure if it's safe to say it's only use for cpu_util. that said,
cpu_util ideally shouldn't be a transform anyways. see the work Avi was
> IPC is one of the
> standard use cases for message queues. the concept of using queues to
> pass around and distribute work is essentially what it's designed for.
> if rabbit or any message queue service can't provide this function, it
> does worry me.
> I see your point here, but Ceilometer aims to take care of the
> OpenStack, monitor it's state. Now it is known as a "Rabbit killer". We
> cannot ignore that if we want anybody uses Ceilometer.
what is the message load we're seeing here? how is your MQ configured?
do you have batching? how many agents/queues do you have? i think this
needs to be reviewed first to be honest as there really isn't much to go on?
More information about the OpenStack-dev