Open Stack

I think most are missing the point a bit.  The question that should really
be asked is, what is right for Swift to continue to scale.  Since the
inception of Openstack, Swift has had to solve for problems of scale that
generally are not shared with the rest of Openstack.

When we first set out to write Swift, we had set, what we thought at the
time were pretty lofty goals for ourselves:

* 100 Billion objects
* 100 Petabytes of data
* 100 K requests/second
* 100 Gb/s throughput

We started with Python figuring that when we hit major bottlenecks, we
would look at other options.  We have been surprised at how far we have
been able to push Python and have met most if not all of the goals above.

As we look toward the future, we realize that we are now looking for how we
will support trillions of objects, 100's of petabytes to exabytes of data,
etc.  We feel that we have finally hit that point that we need more than
incremental improvements, and that we are running out of incremental
improvements that can be made with Python.

What started as a simple experiment by Mike Barton, has turned into quite a
significant improvement in performance and builds a base that can be built
off of for future improvements.  This wasn't built because of it being
"shiny" but out of direct need, and is currently being tested with great
results on production workloads.

I applaud the team that has worked on this at Rackspace, and hope the
community can look at the current needs of Swift, and the merits of the
work that has been accomplished, rather than the politics of "shiny".

Thanks,

--
Chuck


On Thu, Apr 30, 2015 at 11:45 AM John Dickinson <me at not.mn> wrote:

> Swift is a scalable and durable storage engine for storing unstructured
> data. It's been proven time and time again in production in clusters all
> over the world.
>
> We in the Swift developer community are constantly looking for ways to
> improve the codebase and deliver a better quality codebase to users
> everywhere. During the past year, the Rackspace Cloud Files team has been
> exploring the idea of reimplementing parts of Swift in Go. Yesterday, they
> released some of this code, called "hummingbird", for the first time. It's
> been proposed to a "feature/hummingbird" branch in Swift's source repo.
>
> https://review.openstack.org/#/c/178851
>
> I am very excited about this work being in the greater OpenStack Swift
> developer community. If you look at the patch above, you'll see that there
> are various parts of Swift reimplemented in Go. During the next six months
> (i.e. before Tokyo), I would like us to answer this question:
>
> What advantages does a compiled-language object server bring, and do they
> outweigh the costs of using a different language?
>
> Of course, there are a ton of things we need to explore on this topic, but
> I'm happy that we'll be doing it in the context of the open community
> instead of behind closed doors. We will have a fishbowl session in
> Vancouver on this topic. I'm looking forward to the discussion.
>
>
> --John
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150507/445fc89f/attachment.html>