Open Stack

On Thu, May 7, 2015 at 1:10 PM, Chuck Thier <cthier at gmail.com> wrote:

> I think most are missing the point a bit.  The question that should really
> be asked is, what is right for Swift to continue to scale.  Since the
> inception of Openstack, Swift has had to solve for problems of scale that
> generally are not shared with the rest of Openstack.
>
> When we first set out to write Swift, we had set, what we thought at the
> time were pretty lofty goals for ourselves:
>
> * 100 Billion objects
> * 100 Petabytes of data
> * 100 K requests/second
> * 100 Gb/s throughput
>
> We started with Python figuring that when we hit major bottlenecks, we
> would look at other options.  We have been surprised at how far we have
> been able to push Python and have met most if not all of the goals above.
>
> As we look toward the future, we realize that we are now looking for how
> we will support trillions of objects, 100's of petabytes to exabytes of
> data, etc.  We feel that we have finally hit that point that we need more
> than incremental improvements, and that we are running out of incremental
> improvements that can be made with Python.
>
> What started as a simple experiment by Mike Barton, has turned into quite
> a significant improvement in performance and builds a base that can be
> built off of for future improvements.  This wasn't built because of it
> being "shiny" but out of direct need, and is currently being tested with
> great results on production workloads.
>

Out of curiosity, Do you have any numbers to quantify the improvement?


>
> I applaud the team that has worked on this at Rackspace, and hope the
> community can look at the current needs of Swift, and the merits of the
> work that has been accomplished, rather than the politics of "shiny".
>
> Thanks,
>
> --
> Chuck
>
>
> On Thu, Apr 30, 2015 at 11:45 AM John Dickinson <me at not.mn> wrote:
>
>> Swift is a scalable and durable storage engine for storing unstructured
>> data. It's been proven time and time again in production in clusters all
>> over the world.
>>
>> We in the Swift developer community are constantly looking for ways to
>> improve the codebase and deliver a better quality codebase to users
>> everywhere. During the past year, the Rackspace Cloud Files team has been
>> exploring the idea of reimplementing parts of Swift in Go. Yesterday, they
>> released some of this code, called "hummingbird", for the first time. It's
>> been proposed to a "feature/hummingbird" branch in Swift's source repo.
>>
>> https://review.openstack.org/#/c/178851
>>
>> I am very excited about this work being in the greater OpenStack Swift
>> developer community. If you look at the patch above, you'll see that there
>> are various parts of Swift reimplemented in Go. During the next six months
>> (i.e. before Tokyo), I would like us to answer this question:
>>
>> What advantages does a compiled-language object server bring, and do they
>> outweigh the costs of using a different language?
>>
>> Of course, there are a ton of things we need to explore on this topic,
>> but I'm happy that we'll be doing it in the context of the open community
>> instead of behind closed doors. We will have a fishbowl session in
>> Vancouver on this topic. I'm looking forward to the discussion.
>>
>>
>> --John
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150508/b3486902/attachment.html>