Open Stack

On 02/20/2015 10:39 AM, Sean Lynn wrote:
> We finished upgrading to Juno about the time you guys did.  Just checked
> logs across all environments since the time of the Juno upgrade and I'm
> *not* seeing the same errors.
>
> For comparison here's what we have (mostly out-of-the-box):
>
>     api_workers and rpc_workers = 32
>     metadata_workers = 32
>     url_timeout = 30
>     oslo version = 1.4.1
>
> Any related errors in the Neutron logs?
> Couple seemingly dumb questions related to system limits, but:
>
>  1. Could this be a file descriptors limit for the neutron processes?
>  2. Recently we ran into the file descriptors limit in MySQL which
>     showed up with "sporadic but frequent errors" in Neutron.  Under
>     load is your MySQL fd limit being hit?

I'm not familiar with file descriptor limit issues with MySQL. Do you 
mean max_connections issues?

Best,
-jay

>  3. Similar limit question for RabbitMQ.
>
> Let me know if you want any more comparison info.
>
> Sean Lynn
> Time Warner Cable, Inc.
>
>
>
> ----------------------
>
> Kris G. Lindgren:
>
> "
>
>     After our icehouse -> juno upgrade we are noticing sporadic but frequent errors from nova-metadata when trying to serve metadata requests.  The error is the following:
>
>     [req-594325c6-44ed-465c-a8e4-bd5a8e5dbdcb None] Failed to get metadata for ip: x.x.x.x 2015-02-19 12:16:45.903 25007 TRACE nova.api.metadata.handler Traceback (most recent call last): 2015-02-19 12:16:45.903 25007 TRACE nova.api.metadata.handler File /usr/lib/python2.6/site-packages/nova/api/metadata/handler.py, line 150, in _handle_remote_ip_request 2015-02-19 12:16:45.903 25007 TRACE nova.api.metadata.handler meta_data = self.get_metadata_by_remote_address(remote_address) 2015-02-19 12:16:45.903 25007 TRACE nova.api.metadata.handler File /usr/lib/python2.6/site-packages/nova/api/metadata/handler.py, line 82, in get_metadata_by_remote_address 2015-02-19 12:16:45.903 25007 TRACE nova.api.metadata.handler data = base.get_metadata_by_address(self.conductor_api, address)
>
>     ...
>
>     We have increased the number of neutron workers (40 API and 40 RPC), the Neutron url_timeout interval in nova from 30 to 60 seconds. We are only seeing this issue in production or pre-prod environments are fine.
>
>     Is anyone else noticing this or frequent read timeouts when talking to neutron?  Have you found a solution?  What have you tried?
>
>     I am thinking of updating a bunch of the oslo (db, messaging, ect ect) packages to the latest versions to see if things get better.
>
> "
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>