Open Stack

On 4/16/2015 12:27 PM, Jay Pipes wrote:
> On 04/16/2015 09:54 AM, Sean Dague wrote:
>> On 04/16/2015 05:20 PM, Qiming Teng wrote:
>>>
>>> Wondering if there is something misconfigured in my devstack
>>> environment, which was reinstalled on RHEL7 about 10 days ago.
>>> I'm often running into mysql connections problem as shown below:
>>>
>>> $ mysql
>>> ERROR 1040 (HY000): Too many connections
>>>
>>> When I try dump the mysql connection list, I'm getting the followng
>>> result after a 'systemctl restart mariadb.service':
>>>
>>> $ mysqladmin processlist | grep nova | wc -l
>>> 125
>>>
>>> Most of the connections are at Sleep status:
>>>
>>> $ mysqladmin processlist | grep nova | grep Sleep | wc -l
>>> 123
>>>
>>> As for the workload, I'm currently only running two VMs in a multi-host
>>> devstack environment.
>>>
>>> So, my questions:
>>>
>>>    - Why do we have so many mysql connections from nova?
>>>    - Is it possible this is caused some misconfigurations?
>>>    - 125 connections in such a toy setup is insane, any hints on nailing
>>>      down the connections to the responsible nova components?
>>>
>>> Thanks.
>>>
>>> Regards,
>>>    Qiming
>>
>> No, that's about right. It's 1 connection per worker. By default most
>> daemons start 1 worker per processor. Each OpenStack service has a bunch
>> of daemons. It all adds up pretty quick.
>
> And just to add to what Sean says above, there's nothing inherently
> wrong with sleeping connections to MySQL. What *is* wrong, however, is
> that the default max_connections setting in my.cnf is 150. :( I
> frequently recommend upping that to 2000 or more on any modern hardware
> or decent sized VM.
>
> Best,
> -jay
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

What do you consider a decent sized VM?  In devstack we default 
max_connections for postgresql to 200 because we were having connection 
timeout failures in the gate for pg back in the day:

http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/databases/postgresql#n15

But we don't change this for mysql:

http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/databases/mysql

I think the VMs in the gate are running 8 VCPU + 8 GB RAM, not sure 
about disk.

-- 

Thanks,

Matt Riedemann