[openstack-dev] [infra][Neutron] Running out of memory on gate for linux bridge job

Jens Rosenboom j.rosenboom at x-ion.de
Wed Jan 18 10:53:46 UTC 2017


2017-01-13 17:56 GMT+01:00 Clark Boylan <cboylan at sapwetik.org>:
> On Fri, Jan 13, 2017, at 07:48 AM, Jakub Libosvar wrote:
>> Does anybody know whether we can bump memory on nodes in the gate
>> without losing resources for running other jobs?
>> Has anybody experience with memory consumption being higher when using
>> linux bridge agents?
>>
>> Any other ideas?
>
> Ideally I think we would see more work to reduce memory consumption.
> Heat has been able to more than halve their memory usage recently [0].
> Perhaps start by identifying the biggest memory hogs and go from there?
>
> [0]
> http://lists.openstack.org/pipermail/openstack-dev/2017-January/109748.html

In order to have some real data, I've run reproduce.sh for a random
full tempest check and aggregated the memory usage from ps output
during the tempest run [1].
To me it looks like the times of 2G are long gone, Nova is using
almost 2G all by itself. And 8G may be getting tight if additional
stuff like Ceph is being added.

As a side note, we are seeing consistent failures for the Chef
OpenStack Cookbook integration tests on infra. We have set up an
external CI now running on 12G instances and are getting successful
results there. [2]

[1] http://paste.openstack.org/show/595348/
[2] https://review.openstack.org/409900



More information about the OpenStack-dev mailing list