Open Stack

On Wed, Apr 1, 2015 at 8:20 AM, Ian Wienand <iwienand at redhat.com> wrote:

> On 03/27/2015 08:47 PM, Alan Pevec wrote:
>
>> But how come that same recent pyOpenSSL doesn't consume more memory on
>> Ubuntu?
>>
>
> Just to loop back on the final status of this ...
>
> pyOpenSSL 0.14 does seem to use about an order of magnitude more
> memory than 0.13 (2mb -> 20mb).  For details see [1].
>
> This is due to the way it now goes through "cryptography" (the
> package, not the concept :) which binds to openssl using cffi.  This
> ends up parsing a bunch of C to build up the ABI representation, and
> it seems pycparser's model of this consumes most of the memory [2].
> If that is a bug or not remains to be seen.
>
> Ubuntu doesn't notice this in our CI environment because it comes with
> python-openssl 0.13 pre-installed in the image.  Centos started
> hitting this when I merged my change to start using as many libraries
> from pip as possible.
>
> I have a devstack workaround for centos out (preinstall the package)
> [3] and I think a global solution of avoiding it in requirements [4]
> (reviews appreciated).
>
> I'm also thinking about how we can better monitor memory usage for
> jobs.  Being able to see exactly what change pushed up memory usage by
> a large % would have made finding this easier.  We keep some overall
> details for devstack runs in a log file, but there is room to do
> better.
>

Interesting debug, and good to see this was finally nailed.

Few questions:

1) So why did this happen on rax VM only, the same (Centos job)on hpcloud
didn't seem to hit it even when we ran hpcloud VM with 8GB memory.

2) Should this also be sent to centos-devel folks so that they don't
upgrade/update
the pyopenssl in their distro repos until the issue is resolved ?

thanx,
deepak



>
> -i
>
> [1] https://etherpad.openstack.org/p/oom-in-rax-centos7-CI-job
> [2] https://github.com/eliben/pycparser/issues/72
> [3] https://review.openstack.org/168217
> [4] https://review.openstack.org/169596
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150401/0544cb2a/attachment.html>