Open Stack

On 09/04/2014 01:15 PM, Chris Dent wrote:
> On Thu, 4 Sep 2014, Flavio Percoco wrote:
> 
> Thanks for writing this up, interesting read.
> 

Thank you for your feedback :)

Some comments in-line.

>> 5. Ceilometer's recommended storage driver is still MongoDB, although
>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm
>> wrong).
> 
> For sake of reference: Yes, MongoDB is currently the recommended
> store and yes, sqlalchemy support is present. Until recently only
> sqlalchemy support was tested in the gate. Two big changes being
> developed in Juno related to storage:
> 
> * Improved read and write performance in the sqlalchemy setup.
> * time series storage and Gnocchi:
>  
> https://julien.danjou.info/blog/2014/openstack-ceilometer-the-gnocchi-experiment

Awesome, thanks for clarifying this.

[snip]

>> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
>> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
>> flavor on top. [0]
> 
> In my efforts to track this stuff I remain confused on the points in
> these two questions:
> 
> https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#How_does_Zaqar_compare_to_oslo.messaging.3F
> 
> https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#Is_Zaqar_an_under-cloud_or_an_over-cloud_service.3F
> 
> 
> What or where is the boundary between Zaqar and existing messaging
> infrastructure? Not just in terms of technology but also use cases?
> The answers above suggest its not super solid on the use case side,
> notably: "In addition, several projects have expressed interest in
> integrating with Zaqar in order to surface events..."
> 
> Instead of Zaqar doing what it does and instead of oslo.messaging
> abstracting RPC, why isn't the end goal a multi-tenant, multi-protocol
> event pool? Wouldn't that have the most flexibility in terms of
> ecosystem and scalability?

If we put both features, multi-tenancy and multi-protocol, aside for a
bit, we can simplify Zaqars goal down to a "messaging service for the
cloud". I believe this is exactly where the line between Zaqar and other
*queuing* technologies should be drawn. Zaqar is, at the very end, a
messaging service thought for the cloud whereas existing queuing
technologies were not designed for it. By cloud I don't mean
performance, scalability nor anything like that. I'm talking about
providing a service that end-users of the cloud can consume.

The fact that Zaqar is also ideal for the under-cloud is a plus. The
service has been designed to suffice a set of messaging features that
serve perfectly use-cases in both the under-cloud and over-cloud.

If we add to that a multi-protocol transport layer with support for
multi-tenancy, you'll get a queuing service that fits the need of cloud
providers and covers a broader set of use cases like, say, IoT.

I forgot to add this link[0] to my previous email. Does the overview of
the service, the key features and scope help clearing things out a bit?

Please, let me know if they don't. I'm happy to provide more info if needed.

[0] https://wiki.openstack.org/wiki/Zaqar#Overview

> 
>> In addition to the aforementioned concerns and comments, I also would
>> like to share an etherpad that contains some use cases that other
>> integrated projects have for Zaqar[0]. The list is not exhaustive and
>> it'll contain more information before the next meeting.
>>
>> [0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases
> 
> For these, what is Zaqar providing that oslo.messaging (and its
> still extant antecedents) does not? I'm not asking to naysay Zaqar,
> but to understand more clearly what's going on. My interest here comes
> from a general interest in now events and notifications are handled
> throughout OpenStack.

One of the reasons you would want to use Zaqar instead of oslo.messaging
for, say, guest-agents is that you don't want guest-agents to talk to
your main messaging layer. Zaqar will help guest-agents to communicate
with the main service in a more secure, authenticated and isolated way.

If you were going to do that with oslo.messaging, you'd need to have
separate virtual_hosts, exchanges and probably even users. This things
cannot be easily configured without manual intervention. With Zaqar you
can easily rely on your deployed cloud services - keystone, Barbican and
Zaqar, for example - to achieve such isolation and security.

There are also other aspects that are worrisome of relying on the main
messaging infrastructure for the use cases mentioned in that etherpad.
For example, using OpenStack's main rabbitmq instance to communicate
with guest-agents would increase the workload on the infrastructure,
which would require a better scaling strategy for it.

I hope the above clears your doubts. Thanks a lot for your feedback,
it's useful to keep the discussion going and helps everyone to keep
re-evaluating the goals and scopes of the project.

I hope other folks from the team will also chime in and share their
thoughts.

Cheers,
Flavio

-- 
@flaper87
Flavio Percoco