[openstack-dev] OpenStack + PyPy: Status and goals

Alex Gaynor alex.gaynor at gmail.com
Sun Sep 15 21:57:38 UTC 2013


The short answer is, PyPy has its own implementation of greenlets,
https://bitbucket.org/pypy/pypy/src/default/lib_pypy/greenlet.py , this is
built on top of a module called _continuation,
http://doc.pypy.org/en/latest/stackless.html contains some of the details
of how it works.

Alex


On Sun, Sep 15, 2013 at 2:43 PM, Joshua Harlow <harlowja at yahoo-inc.com>wrote:

>  Cool,
>
> Are there any technical docs for how eventlet/greenlet work in PyPy,
>
> From my knowledge of greenlet its doing some pretty low level stuff that
> would seem hard to mirror in PyPy.
>
> https://github.com/python-greenlet/greenlet/blob/master/greenlet.c#L9
>
> And the very platform specific stack switching functions @
> https://github.com/python-greenlet/greenlet/tree/master/platform
>
> It would seem like greenlet is pretty tightly coupled to the cpython and
> its functionality. I'd like to know how it can operate in pypy without
> major modifications, maybe u know of such a documentation that explains
> this. I'd be interested in reading that at least (maybe others would like
> to also).
>
> :-)
>
> -Josh
> ------------------------------
> *From:* Alex Gaynor [alex.gaynor at gmail.com]
> *Sent:* Tuesday, September 10, 2013 8:18 AM
> *To:* OpenStack Development Mailing List
> *Subject:* Re: [openstack-dev] OpenStack + PyPy: Status and goals
>
>   Hi Roman,
>
>  Yes eventlet works well on PyPy, both Marconi and Swift use it.
>
>  Alex
>
>
> On Mon, Sep 9, 2013 at 10:15 PM, Roman Podolyaka <rpodolyaka at mirantis.com>wrote:
>
>> Hi Alex,
>>
>>  That's really cool! I believe, performance is not the only benefit we
>> can get from running OpenStack projects on PyPy. We  can also improve the
>> overall "correctness" of our code (as PyPy behaves differently with
>> non-closed files, etc), just like compiling of your C/C++ app using
>> different compilers can show hidden errors.
>>
>>  And what about eventlet? Does it work well on PyPy? (as it is used in
>> Nova, Neutron, etc)
>>
>>  Thanks,
>> Roman
>>
>>
>>  On Tue, Sep 10, 2013 at 12:28 AM, Alex Gaynor <alex.gaynor at gmail.com>wrote:
>>
>>>   Hi all,
>>>
>>>  Many of you have probably seen me send review requests in the last few
>>> weeks
>>> about adding PyPy support to various OpenStack projects. A few people
>>> were
>>> confused by these, so I wanted to fill everyone in on what I'm up to :)
>>>
>>>  First, for those who aren't familiar with what PyPy is: PyPy is an
>>> implementation of the Python language which includes a high performance
>>> tracing
>>> just-in-time compiler and which is faster than CPython (the reference,
>>> and most
>>> widely deployed, implementation) on almost all workloads.
>>>
>>>  The current status is:
>>>
>>>  Two major projects work, both Marconi and Swift, Marconi is gating
>>> against PyPy
>>> already, Swift isn't yet since I needed to fix a few small PyPy bugs and
>>> those
>>> aren't in a release yet, expect it soon :)
>>>
>>>  In terms of results, I've observed 30% performance improvements on GET
>>> workloads for Swift under PyPy vs. CPython (other workloads haven't been
>>> benchmarked tet). I believe the Marconi folks have also observed some
>>> performance wins, but I'll let them speak to that, I don't have the full
>>> details.
>>>
>>>  Many python-clients projects are also working out of the box and
>>> gating:
>>> including novaclient, swiftclient, marconiclient, ceilometerclient,
>>> heatclient,
>>> and ironicclient!
>>>
>>>  There's a few outstanding reviews to add PyPy gating for cinderclient,
>>> troveclient, and glanceclient.
>>>
>>>  In terms of future direction:
>>>
>>>  I'm going to continue to work on getting more projects running and
>>> gating
>>> against PyPy.
>>>
>>>  Right now I'm focusing a lot of my attention on improving Swift
>>> performance,
>>> particularly under PyPy, but also under CPython.
>>>
>>>  I'm hoping some day PyPy will be the default way to deploy OpenStack!
>>>
>>>
>>>  If you're interested in getting your project running on PyPy, or
>>> looking at
>>> performance under it, please let me know, I'm always interested in
>>> helping!
>>>
>>>  Thanks,
>>> Alex
>>>
>>>  --
>>> "I disapprove of what you say, but I will defend to the death your right
>>> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>> "The people's good is the highest law." -- Cicero
>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>>
>>>  _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
>  --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130915/e98c4802/attachment.html>


More information about the OpenStack-dev mailing list