Open Stack

Chuck (and/or others who understand tor have experienced the limits of
Python)

I found this comment of yours incredibly intriguing: "we are running out of
incremental improvements that can be made with Python".

Given your work with Swift thus far, what sort of limitations have you
discovered that had to do specifically with the fact we're using Python? I
haven't run into real-life limitations specific to a particular language
before (I usually run into issues with my approach rather than limitations
with the language itself) so i find this to be a fascinating (and perhaps
accidental) consideration.



*Adam Lawson*

AQORN, Inc.
427 North Tatnall Street
Ste. 58461
Wilmington, Delaware 19801-2230
Toll-free: (844) 4-AQORN-NOW ext. 101
International: +1 302-387-4660
Direct: +1 916-246-2072


On Thu, May 7, 2015 at 3:48 PM, Clint Byrum <clint at fewbar.com> wrote:

> Excerpts from Chuck Thier's message of 2015-05-07 13:10:13 -0700:
> > I think most are missing the point a bit.  The question that should
> really
> > be asked is, what is right for Swift to continue to scale.  Since the
> > inception of Openstack, Swift has had to solve for problems of scale that
> > generally are not shared with the rest of Openstack.
> >
> > When we first set out to write Swift, we had set, what we thought at the
> > time were pretty lofty goals for ourselves:
> >
> > * 100 Billion objects
> > * 100 Petabytes of data
> > * 100 K requests/second
> > * 100 Gb/s throughput
> >
> > We started with Python figuring that when we hit major bottlenecks, we
> > would look at other options.  We have been surprised at how far we have
> > been able to push Python and have met most if not all of the goals above.
> >
> > As we look toward the future, we realize that we are now looking for how
> we
> > will support trillions of objects, 100's of petabytes to exabytes of
> data,
> > etc.  We feel that we have finally hit that point that we need more than
> > incremental improvements, and that we are running out of incremental
> > improvements that can be made with Python.
> >
> > What started as a simple experiment by Mike Barton, has turned into
> quite a
> > significant improvement in performance and builds a base that can be
> built
> > off of for future improvements.  This wasn't built because of it being
> > "shiny" but out of direct need, and is currently being tested with great
> > results on production workloads.
> >
> > I applaud the team that has worked on this at Rackspace, and hope the
> > community can look at the current needs of Swift, and the merits of the
> > work that has been accomplished, rather than the politics of "shiny".
> >
>
> Chuck, much respect to you and the team for everything accomplished.
>
> I'm still very curious to hear if anybody has been willing to try to
> make Swift work on pypy. This is pretty much why pypy exists, and making
> pypy work for Swift could mean some really nice residual benefits to the
> other projects that haven't gone as far as to experiment with a compiled
> language like Go yet. There's also the other benefit that pypy would
> gain some eyeballs and improvements that we could feed back into it.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150507/7b5565a2/attachment.html>