Open Stack

On 09/04/2014 07:08 PM, Clint Byrum wrote:
> Excerpts from Flavio Percoco's message of 2014-09-04 06:01:45 -0700:
>> On 09/04/2014 02:14 PM, Sean Dague wrote:
>>> On 09/04/2014 03:08 AM, Flavio Percoco wrote:
>>>> Greetings,
>>>>
>>>> Last Tuesday the TC held the first graduation review for Zaqar. During
>>>> the meeting some concerns arose. I've listed those concerns below with
>>>> some comments hoping that it will help starting a discussion before the
>>>> next meeting. In addition, I've added some comments about the project
>>>> stability at the bottom and an etherpad link pointing to a list of use
>>>> cases for Zaqar.
>>>>
>>>> # Concerns
>>>>
>>>> - Concern on operational burden of requiring NoSQL deploy expertise to
>>>> the mix of openstack operational skills
>>>>
>>>> For those of you not familiar with Zaqar, it currently supports 2 nosql
>>>> drivers - MongoDB and Redis - and those are the only 2 drivers it
>>>> supports for now. This will require operators willing to use Zaqar to
>>>> maintain a new (?) NoSQL technology in their system. Before expressing
>>>> our thoughts on this matter, let me say that:
>>>>
>>>>     1. By removing the SQLAlchemy driver, we basically removed the chance
>>>> for operators to use an already deployed "OpenStack-technology"
>>>>     2. Zaqar won't be backed by any AMQP based messaging technology for
>>>> now. Here's[0] a summary of the research the team (mostly done by
>>>> Victoria) did during Juno
>>>>     3. We (OpenStack) used to require Redis for the zmq matchmaker
>>>>     4. We (OpenStack) also use memcached for caching and as the oslo
>>>> caching lib becomes available - or a wrapper on top of dogpile.cache -
>>>> Redis may be used in place of memcached in more and more deployments.
>>>>     5. Ceilometer's recommended storage driver is still MongoDB, although
>>>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).
>>>>
>>>> That being said, it's obvious we already, to some extent, promote some
>>>> NoSQL technologies. However, for the sake of the discussion, lets assume
>>>> we don't.
>>>>
>>>> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
>>>> keep avoiding these technologies. NoSQL technologies have been around
>>>> for years and we should be prepared - including OpenStack operators - to
>>>> support these technologies. Not every tool is good for all tasks - one
>>>> of the reasons we removed the sqlalchemy driver in the first place -
>>>> therefore it's impossible to keep an homogeneous environment for all
>>>> services.
>>>>
>>>> With this, I'm not suggesting to ignore the risks and the extra burden
>>>> this adds but, instead of attempting to avoid it completely by not
>>>> evolving the stack of services we provide, we should probably work on
>>>> defining a reasonable subset of NoSQL services we are OK with
>>>> supporting. This will help making the burden smaller and it'll give
>>>> operators the option to choose.
>>>>
>>>> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
>>>
>>> I've been one of the consistent voices concerned about a hard
>>> requirement on adding NoSQL into the mix. So I'll explain that thinking
>>> a bit more.
>>>
>>> I feel like when the TC makes an integration decision previously this
>>> has been about evaluating the project applying for integration, and if
>>> they met some specific criteria they were told about some time in the
>>> past. I think that's the wrong approach. It's a locally optimized
>>> approach that fails to ask the more interesting question.
>>>
>>> Is OpenStack better as a whole if this is a mandatory component of
>>> OpenStack? Better being defined as technically better (more features,
>>> less janky code work arounds, less unexpected behavior from the stack).
>>> Better from the sense of easier or harder to run an actual cloud by our
>>> Operators (taking into account what kinds of moving parts they are now
>>> expected to manage). Better from the sense of a better user experience
>>> in interacting with OpenStack as whole. Better from a sense that the
>>> OpenStack release will experience less bugs, less unexpected cross
>>> project interactions, an a greater overall feel of consistency so that
>>> the OpenStack API feels like one thing.
>>>
>>> https://dague.net/2014/08/26/openstack-as-layers/
>>>
>>> One of the interesting qualities of Layers 1 & 2 is they all follow an
>>> AMQP + RDBMS pattern (excepting swift). You can have a very effective
>>> IaaS out of that stack. They are the things that you can provide pretty
>>> solid integration testing on (and if you look at where everything stood
>>> before the new TC mandates on testing / upgrade that was basically what
>>> was getting integration tested). (Also note, I'll accept Barbican is
>>> probably in the wrong layer, and should be a Layer 2 service.)
>>>
>>> While large shops can afford to have a dedicated team to figure out how
>>> to make mongo or redis HA, provide monitoring, have a DR plan for when a
>>> huricane requires them to flip datacenters, that basically means
>>> OpenStack heads further down the path of "only for the big folks". I
>>> don't want OpenStack to be only for the big folks, I want OpenStack to
>>> be for all sized folks. I really do want to have all the local small
>>> colleges around here have OpenStack clouds, because it's something that
>>> people believe they can do and manage. I know the people that work in
>>> this places, they all come out to the LUG I run. We've talked about
>>> this. OpenStack is basically seen as too complex for them to use as it
>>> stands, and that pains me a ton.
>>>
>>> So I think Zaqar is good software, and really useful part of our
>>> ecosystem, but this added step function burden of a 3rd class of support
>>> software that has to be maintained... seems like it takes us further
>>> away from OpenStack at a small scale. If we were thinking about Zaqar as
>>> a thing that we could replace olso.messaging with, that becomes
>>> interesting in a different way, because we could instead of having 3
>>> classes of support software, remain at 2, just take a sideways shift on
>>> one of them. But that's not actually the path we are on.
>>>
>>> So, honestly, I'll probably remain -1 on the final integration vote, not
>>> because Zaqar is bad, but because I'm feeling more firmly that for
>>> OpenStack to not leave the small deployers behind we need to redefine
>>> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
>>> parts of my diagram, and consider the rest of the layers exciting parts
>>> of our ecosystem that more advanced users may choose to deploy to meet
>>> their needs. Smaller tent, big ecosystem, easier on ramp.
>>>
>>> I realize that largely means Zaqar would be caught up in a definition
>>> discussion outside of it's control, and that's kind of unfortunate, as
>>> Flavio and team have been doing a bang up job of late. But we need to
>>> stop considering "integration" as the end game of all interesting
>>> software in the OpenStack ecosystem, and I think it's better to have
>>> that conversation sooner rather than later.
>>>
>>> Anyway, my $0.02.
>>
>> I really like your perspective and vision for OpenStack. I think aiming
>> to make OpenStack more accessible for small teams and deployments is
>> something we all should keep in mind and strive for.
>>
>> IIUC, your -1 stays because you think Zaqar is not an essential part of
>> OpenStack and not necessarily because Zaqar requires a NoSQL technology
>> - the later is just an addition to your main concern. But even if your
>> concern was just about the NoSQL technology, I still wonder:
>>
>> Why does it have to be all or nothing?
>>
>> Technically speaking, deployers can install *just* the services they
>> need. These services may or may not depend on other services but there's
>> to be a balance between simplifying deployments and making them richer.
>> For example, I could argue saying that deploying Zaqar is very cheap,
>> you just need to install Zaqar, pick a storage and you're done. You
>> don't need any of the other services because even keystone is optional
>> if you don't care about authentication. This probably wouldn't be
>> considered an "OpenStack Cloud" but it still is a cloud service that is
>> part of - or at least was born as part of - OpenStack.
>>
> 
> "Just install it" is a pretty tall order. Monitoring, backups, DR,
> orchestration, availability, etc. Every new thing we ask everyone to
> install adds complexity and thus cost.

I know I oversimplified it by saying "just install it". I did that on
purpose to make a point - should have probably mentioned this. The point
I was getting at is the one you actually mentioned below - definitely
better than me. :D


> OpenStack contributors and operators are quite interested in
> portability. Some operators will want to operate a small cloud and then
> inject some of their work into a much larger cloud. If, however, their
> small cloud is lacking Zaqar, and the big one is not, then they cannot
> use Zaqar on the bigger cloud. This is bad for the bigger cloud. The
> big cloud operator has it in their best interest to make the small cloud
> more consumable to seed their section of the market.

I think this could actually happen the other way around as well.
Furthermore, this can - or actually does - happen with the existing
integrated services.

> 
> What is under debate right now, is how much of that extra stuff does
> OpenStack want to champion versus enable. If OpenStack says "Zaqar is
> our messaging thing, deploy it!" Cloud operators who don't will have to
> deal with the lack of portability of apps that rely on it to their cloud.

Would not adding new projects fix this issue?

Operators who don't deploy Zaqar will have to deal with the lack of
portability from clouds that do have/use Zaqar, regardless Zaqar is
championed as a messaging service for OpenStack.

> 
> So, by not championing something optional, we lessen the burden on
> OpenStack operators to support new technologies, and we open up the space
> for fragmentation. There's no one right answer IMO. In spaces like that,
> I don't think we should succumb to the subjective arguments and just pull
> something in. Instead, I'd like to see us weigh the options and come up
> with a third option for when we can't choose between "in" and "out".

Totally I agree on the fact that we should have a third option, group,
whatever. What I'm arguing, though, is that I don't think this should
hold Zaqar's integration back. Integrating a project is not forever.
Thousands of things could happen along the way and what I expect to
happen during Kilo is a re-organization/re-labeling - not sure if
there's a better term for that - of those projects we consider
*optional* in our integrated release.

Re-labeling the project won't hurt neither OpenStack's nor the project's
 development. Holding back Zaqar's graduation on this will affect its
adoption quite a bit - as it has already done.

Flavio

-- 
@flaper87
Flavio Percoco