Open Stack

My responses are inline and to question 5, which, like you, I think is the key.

On 25 May 2016, at 3:48, Sean Dague wrote:

> I've been watching the threads, trying to digest, and find the way's
> this is getting sliced doesn't quite slice the way I've been thinking
> about it. (which might just means I've been thinking about it wrong).
> However, here is my current set of thoughts on things.
>
> 1. Should OpenStack be open to more languages?
>
> I've long thought the answer should be yes. Especially if it means we
> end up with keystonemiddleware, keystoneauth, oslo.config in other
> languages that let us share elements of infrastructure pretty
> seamlessly. The OpenStack model of building services that register in a
> service catalog and use common tokens for permissions through a bunch of
> services is quite valuable. There are definitely people that have Java
> applications that fit into the OpenStack model, but have no place to
> collaborate on them.
>
> (Note: nothing about the current proposal goes anywhere near this)
>
> 2. Is Go a "good" language to add to the community?
>
> Here I am far more mixed. In programming language time, Go is super new.
> It is roughly the same age as the OpenStack project. The idea that Go and
> Python programmers overlap seems to be because some shops that used
> to do a lot in Python, now do some things in Go.
>
> But when compared to other languages in our bag, Javascript, Bash. These
> are things that go back 2 decades. Unless you have avoided Linux or the
> Web successfully for 2 decades, you've done these in some form. Maybe
> not being an expert, but there is vestigial bits of knowledge there. So
> they *are* different. In the same way that C or Java are different, for
> having age. The likelihood of finding community members than know Python
> + one of these is actually *way* higher than Python + Go, just based on
> duration of existence. In a decade that probably won't be true.
>
> 3. Are there performance problems where python really can't get there?
>
> This seems like a pretty clear "yes". It shouldn't be surprising. Python
> has no jit (yes there is pypy, but it's compat story isn't here). There
> is a reason a bunch of python libs have native components for speed -
> numpy, lxml, cryptography, even yaml throws a warning that you should
> really compile the native version for performance when there is full
> python fallback.
>
> The Swift team did a very good job demonstrating where these issues are
> with trying to get raw disk IO. It was a great analysis, and kudos to
> that team for looking at so many angles here.
>
> 4. Do we want to be in the business of building data plane services that
> will all run into python limitations, and will all need to be rewritten
> in another language?
>
> This is a slightly different spin on the question Thierry is asking.
>
> Control Plane services are very unlikely to ever hit a scaling concern
> where rewriting the service in another language is needed for
> performance issues. These are orchestrators, and the time spent in them
> is vastly less than the operations they trigger (start a vm, configure a
> switch, boot a database server). There was a whole lot of talk in the
> threads of "well that's not innovative, no one will want to do just
> that", which seems weird, because that's most of OpenStack. And it's
> pretty much where all the effort in the containers space is right now,
> with a new container fleet manager every couple of weeks. So thinking
> that this is a boring problem no one wants to solve, doesn't hold water
> with me.
>
> Data Plane services seem like they will all end up in the boat of
> "python is not fast enough". Be it serving data from disk, mass DNS
> transfers, time series database, message queues. They will all
> eventually hit the python wall. Swift hit it first because of the
> maturity of the project and they are now focused on this kind of
> optimization, as that's what their user base demands. However I think
> all other data plane services will hit this as well.
>
> Glance (which is partially a data plane service) did hit this limit, and
> the way it is largely mitigated by folks is by using Ceph and exposing that
> directly to Nova so now Glance is only in the location game and metadata
> game, and Ceph is in the data plane game.
>
> When it comes to doing data plan services in OpenStack, I'm quite mixed.
> The technology concerns for data plane
> services are quite different. All the control plane services kind of
> look and feel the same. An API + worker model, a DB for state, message
> passing / rpc to put work to the workers. This is a common pattern and
> is something which even for all the project differences, does end up
> kind of common between parts. Projects that follow this model are
> debuggable as a group not too badly.
>
> 5. Where does Swift fit?
>
> This I think has always been a tension point in the community (at least
> since I joined in 2012). Swift is an original service of OpenStack, as
> it started as Swift and Nova. But they were very different things. Swift
> is a data service, Nova was a control plane. Much of what is now
> OpenStack is Nova derivative in some way (some times direct extractions
> (Glance, Cinder, Ironic), some times convergent paths (Neutron). And
> then with that many examples, lots of other things built in similar ways.
>
> Swift doesn't use common oslo components.

We currently use oslopshinx, pbr, and hacking.

> That actually makes debugging
> it quite different compared to the rest of OpenStack. The lack of
> oslo.log means structured JSON log messages to Elastic Search, are not
> a thing.

This seems to always be the tension: Swift doesn't use oslo.log or
oslo.config.

This was covered two years ago[0][1] by the TC. As part of that
conversation, Swift contributors evaluated various oslo components[2].

The end result of the "gap analysis" conversation was, quite simply,
that it was dropped. The issues identified were that Swift doesn't use
oslo.log and oslo.config and that Swift does some releases between the
end-of-cycle integrated releases. Since that conversation, all
OpenStack projects have adopted versioning that mirrors what Swift has
done, several have also been doing intermediate releases (and the
whole concept of an "integrated" release was dropped), and the
disagreement around oslo.log and oslo.config were largely swept under
the rug.

As mentioned in the meeting from two years ago, I still ask at every
summit (and ops summit when possible) about Swift using or not using
these libraries. And the answer hasn't changed. These are quite simply
not pain points for people who are deploying Swift in production and
participating in the upstream community. Because of the large
development cost of integrating these [2], the large amount of felt
pain points and feature gaps we've been working on, and because no
Swift deployer has started asking for it, we have not prioritized
integrating these two libraries. Put simply, they solve a problem that
we don't have, so we've been focusing on other things.

[0] The gap analysis discussion came from the list of requirements
the TC had put on projects that wanted to be incubated (and then
integrated). A checklist was developed, and then every existing
project was evaluated against that list to make sure everyone was in
compliance.

[1] http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-08-05-20.02.log.html
and https://etherpad.openstack.org/p/swift_gap_analysis

[2] https://etherpad.openstack.org/p/swift_gap_scratchpad

> Swift has a very different model in it's service split.

Swift is not an object storage provisioning system. It's an object
storage engine itself. That's different than Nova/Cinder/Neutron, but
it's not too different than other projects like Glance, Keystone, and
others (see the lists that others have made about the various
"dataplane" projects in OpenStack).

That's not to say that Swift isn't pluggable. There's a diverse set of
projects in the wider ecosystem that plug in to and extend Swift's
functionality. A few of the open-source things I've heard of are
listed below, and this doesn't count the various internal, private
things that other companies have done to integrate Swift and
EMC/NetApp/Cleversafe/Scality/etc.

 - Storlets
 - SwiftOnFile
 - swift-on-ceph
 - Swift on Kinetic drives
 - Tape backend
 - Swift on GPFS
 - Swift on HPSS

> Swift doesn't use global requirements.

This is slightly unfair. We do not actively land every update that the
bot proposes, but as pointed out in the Austin summit sessions, we do
our best to ensure that the requirements we have listed don't
conflict. We add any new requirements we have to the global-
requirements repo, and when we need a minimum version bump, we also
push that to global-requirements and land that line in our
requirements file.

If Swift works fine with dependency Foo at v22 or newer, there's no
reason to update it to require at least v25 because another project
needs a new feature introduced in v25. Upgrading the versions causes
deployers a lot of extra packaging work, so we'd prefer to not do that
(we're all deployers too).

> Swift ensures it can run without
> Keystone, because their goal is Swift everywhere, whether or not it's
> part of the rest of OpenStack.

Add in that some Swift deployers also require having multiple auth
systems installed at the same time.

>
> These are all fine goals, but they definitely have led to tensions on
> all sides.
>
> And I think part of the question is "are these tensions that need to be
> solved"...

Frankly, I haven't been seeing/feeling much of the tensions at all in
the past one or two years. I was pretty sure a lot of this was
settled.

Around the same time as the whole TC gap analysis stuff happened, there
was also the big discussion around DefCore. An initial proposal for
DefCore had dropped object storage as a required capability[1], and
the resulting conversation ended up with the different marks we have
today, including the "OpenStack Powered Platform" which does require
Swift.


[1] https://review.openstack.org/#/c/119689/2
  http://lists.openstack.org/pipermail/defcore-committee/2014-September/000349.html
  https://review.openstack.org/#/c/136677/

> or "is this data that this thing is different". Which isn't to
> say that Swift is bad, it's just definitively different than much of the
> ecosystem. Maybe Swift should be graduated beyond OpenStack, because
> it's scope cross cuts much differently. Ceph isn't part of OpenStack,
> but it's in 50% of installs. libvirt isn't part of OpenStack, but it's
> in 90% of installs. And in both of those cases OpenStack is one of the
> biggest drivers of their use.
>
> Which, gets contentious because people feel like this is kicking
> something out. And that I can understand. There is a lot of emotion
> wrapped up in labels and who gets to be on the the OpenStack home page.
> I wish there wasn't. Good software should get deployed because it is
> good and solves a need, not because of labels. I'm not sure Swift users
> really care that Swift is OpenStack. They care that Swift is Swift. And
> Swift being Swift, but not being OpenStack would open up degrees of
> freedom in Swift being more Swift centric without the same friction
> from the rest of of the ecosystem.


Yeah, it does feel like "kicking something out". Being part of
OpenStack does mean something; it's an important label. Companies see
OpenStack as a safe place where they can contribute and cooperate with
competitors. If OpenStack didn't exist (or if a project isn't under
the OpenStack umbrella), it reduces the number of contributors.[1]
Remember all the TC conversations around adding incubated projects and
the debate around "if we add them, then the number of contributors
will go up"? Regardless of if that happened or not, the point is that
even the TC sees the OpenStack label as something that goes beyond a
few technical checkboxes. The original reason for the Big Tent
is to answer the question "are you one of us".

Swift helped found OpenStack. We participate in all the events. We
integrate with other projects. We use the same processes. We do a lot
of cross-project contribution. We're viewed as one of the most stable
projects. We've got a growing contributor base. Sounds to me like
Swift fits the "one of us" criteria.

So when someone proposes a project is removed from the OpenStack
umbrella, it stings. (Not to mention the sugarcoating of calling it
"graduation".)

[1] As a small example of this, the Swift team has recently
migrated two dependencies to the openstack/* namespace simply
because large companies that contribute have already given approval
to contribute to OpenStack projects. Otherwise, these devs need to
get special permission from their corporate lawyers.

>
> ...
>
> Honestly, we probably need to just address #5 first and foremost. It's
> been dodged around for a long time, the tensions haven't gotten any better.
> We have just decided the standoff around Swift being different is going
> to remain.
> The global requirements proposal is a good example of that
> https://review.openstack.org/#/c/88736/ - 434 revisions later.
>
> From that we have one data point on #4. Are we going to be building a
> set of data plane services that can't be in python? If Swift wasn't the
> leading
> example here would we be having the conversation at all?
>
>   -Sean

This whole "debate" started with the Swift team coming to the wider
OpenStack community, announcing our plans for improving our project
and making sure that we all had enough time to solve any cross-project
issues as they may come up.

It seems to have become an existential crisis for Openstack where
we're debating OpenStack's scope and who's in and who's out. What
happened to "getting out of the way"[1]?

So to answer your question "where does Swift fit?", Swift belongs in
OpenStack. It always has, and I hope it always will.

[1] http://ttx.re/stepping-out-of-the-way.html


>
> -- 
> Sean Dague
> http://dague.net
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160525/0cd3c3a3/attachment.pgp>