Open Stack

On 09/04/2014 02:14 PM, Sean Dague wrote:
> On 09/04/2014 03:08 AM, Flavio Percoco wrote:
>> Greetings,
>>
>> Last Tuesday the TC held the first graduation review for Zaqar. During
>> the meeting some concerns arose. I've listed those concerns below with
>> some comments hoping that it will help starting a discussion before the
>> next meeting. In addition, I've added some comments about the project
>> stability at the bottom and an etherpad link pointing to a list of use
>> cases for Zaqar.
>>
>> # Concerns
>>
>> - Concern on operational burden of requiring NoSQL deploy expertise to
>> the mix of openstack operational skills
>>
>> For those of you not familiar with Zaqar, it currently supports 2 nosql
>> drivers - MongoDB and Redis - and those are the only 2 drivers it
>> supports for now. This will require operators willing to use Zaqar to
>> maintain a new (?) NoSQL technology in their system. Before expressing
>> our thoughts on this matter, let me say that:
>>
>> 	1. By removing the SQLAlchemy driver, we basically removed the chance
>> for operators to use an already deployed "OpenStack-technology"
>> 	2. Zaqar won't be backed by any AMQP based messaging technology for
>> now. Here's[0] a summary of the research the team (mostly done by
>> Victoria) did during Juno
>> 	3. We (OpenStack) used to require Redis for the zmq matchmaker
>> 	4. We (OpenStack) also use memcached for caching and as the oslo
>> caching lib becomes available - or a wrapper on top of dogpile.cache -
>> Redis may be used in place of memcached in more and more deployments.
>> 	5. Ceilometer's recommended storage driver is still MongoDB, although
>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).
>>
>> That being said, it's obvious we already, to some extent, promote some
>> NoSQL technologies. However, for the sake of the discussion, lets assume
>> we don't.
>>
>> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
>> keep avoiding these technologies. NoSQL technologies have been around
>> for years and we should be prepared - including OpenStack operators - to
>> support these technologies. Not every tool is good for all tasks - one
>> of the reasons we removed the sqlalchemy driver in the first place -
>> therefore it's impossible to keep an homogeneous environment for all
>> services.
>>
>> With this, I'm not suggesting to ignore the risks and the extra burden
>> this adds but, instead of attempting to avoid it completely by not
>> evolving the stack of services we provide, we should probably work on
>> defining a reasonable subset of NoSQL services we are OK with
>> supporting. This will help making the burden smaller and it'll give
>> operators the option to choose.
>>
>> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> 
> I've been one of the consistent voices concerned about a hard
> requirement on adding NoSQL into the mix. So I'll explain that thinking
> a bit more.
> 
> I feel like when the TC makes an integration decision previously this
> has been about evaluating the project applying for integration, and if
> they met some specific criteria they were told about some time in the
> past. I think that's the wrong approach. It's a locally optimized
> approach that fails to ask the more interesting question.
> 
> Is OpenStack better as a whole if this is a mandatory component of
> OpenStack? Better being defined as technically better (more features,
> less janky code work arounds, less unexpected behavior from the stack).
> Better from the sense of easier or harder to run an actual cloud by our
> Operators (taking into account what kinds of moving parts they are now
> expected to manage). Better from the sense of a better user experience
> in interacting with OpenStack as whole. Better from a sense that the
> OpenStack release will experience less bugs, less unexpected cross
> project interactions, an a greater overall feel of consistency so that
> the OpenStack API feels like one thing.
> 
> https://dague.net/2014/08/26/openstack-as-layers/
> 
> One of the interesting qualities of Layers 1 & 2 is they all follow an
> AMQP + RDBMS pattern (excepting swift). You can have a very effective
> IaaS out of that stack. They are the things that you can provide pretty
> solid integration testing on (and if you look at where everything stood
> before the new TC mandates on testing / upgrade that was basically what
> was getting integration tested). (Also note, I'll accept Barbican is
> probably in the wrong layer, and should be a Layer 2 service.)
> 
> While large shops can afford to have a dedicated team to figure out how
> to make mongo or redis HA, provide monitoring, have a DR plan for when a
> huricane requires them to flip datacenters, that basically means
> OpenStack heads further down the path of "only for the big folks". I
> don't want OpenStack to be only for the big folks, I want OpenStack to
> be for all sized folks. I really do want to have all the local small
> colleges around here have OpenStack clouds, because it's something that
> people believe they can do and manage. I know the people that work in
> this places, they all come out to the LUG I run. We've talked about
> this. OpenStack is basically seen as too complex for them to use as it
> stands, and that pains me a ton.
> 
> So I think Zaqar is good software, and really useful part of our
> ecosystem, but this added step function burden of a 3rd class of support
> software that has to be maintained... seems like it takes us further
> away from OpenStack at a small scale. If we were thinking about Zaqar as
> a thing that we could replace olso.messaging with, that becomes
> interesting in a different way, because we could instead of having 3
> classes of support software, remain at 2, just take a sideways shift on
> one of them. But that's not actually the path we are on.
> 
> So, honestly, I'll probably remain -1 on the final integration vote, not
> because Zaqar is bad, but because I'm feeling more firmly that for
> OpenStack to not leave the small deployers behind we need to redefine
> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
> parts of my diagram, and consider the rest of the layers exciting parts
> of our ecosystem that more advanced users may choose to deploy to meet
> their needs. Smaller tent, big ecosystem, easier on ramp.
> 
> I realize that largely means Zaqar would be caught up in a definition
> discussion outside of it's control, and that's kind of unfortunate, as
> Flavio and team have been doing a bang up job of late. But we need to
> stop considering "integration" as the end game of all interesting
> software in the OpenStack ecosystem, and I think it's better to have
> that conversation sooner rather than later.
> 
> Anyway, my $0.02.

I really like your perspective and vision for OpenStack. I think aiming
to make OpenStack more accessible for small teams and deployments is
something we all should keep in mind and strive for.

IIUC, your -1 stays because you think Zaqar is not an essential part of
OpenStack and not necessarily because Zaqar requires a NoSQL technology
- the later is just an addition to your main concern. But even if your
concern was just about the NoSQL technology, I still wonder:

Why does it have to be all or nothing?

Technically speaking, deployers can install *just* the services they
need. These services may or may not depend on other services but there's
to be a balance between simplifying deployments and making them richer.
For example, I could argue saying that deploying Zaqar is very cheap,
you just need to install Zaqar, pick a storage and you're done. You
don't need any of the other services because even keystone is optional
if you don't care about authentication. This probably wouldn't be
considered an "OpenStack Cloud" but it still is a cloud service that is
part of - or at least was born as part of - OpenStack.

I really don't want to start the core/non-core discussion here. I'm
saying all this from a technical perspective and not a
marketing/legal/whatever one(1). Although I'd love to, I'm not trying to
convince you otherwise but to understand the root of your concern.

In Zaqar, we've decided not to merge store drivers in the core base that
we: 1. don't have the enough knowledge for and 2. we can't maintain. I
believe this is a narrower version of your overall concern. The
difference is that we do have a way to let those drivers exist without
being in the code base.

What I'm trying to get to is that currently, in OpenStack, we don't have
a space for projects that are part of OpenStack but are also optional.
With my OpenStack hat on, I don't think this should stop OpenStack's
growth. We should keep moving forward and re-evaluate projects when such
spaces exists. I'd personally be more than happy to move Zaqar to a
"layer-4-integrated-non-essential-projects" space once it exists but we
need to help projects to come out of the limbo. We need to give big
deployers enough information and proofs to trust the project goals, the
team and the stability of both of them. We've re-evaluate projects more
than once, we ask ourselves the same questions about some projects every
once in a while to verify that everything is aligned with OpenStack's
goals and ecosystem. I think this is a sane thing to do so why shouldn't
we do it with Zaqar too?, for example - I admit it, I just put my
Zaqar's hat on again ;).

In addition to the above, I don't think it should prevent existing
integrated projects to consume Zaqar. I'd rather have integrated
projects consuming other OpenStack services than relying on external
ones, even if they're already deployed.

Sean, thanks for expanding the thoughts on that concern. Your email
helped clearing some things out about it.

Cheers,
Flavio

(1) Please, anyone willing to chime in, do it, but lets avoid
discussions that are not related to Zaqar's integration.

-- 
@flaper87
Flavio Percoco