Open Stack

Hi,

I introduced this in my nova api-paste.ini:

[pipeline:openstack_compute_api_v2]
pipeline = faultwrap authtoken keystonecontext ratelimit osapi_compute_app_v2

[pipeline:openstack_volume_api_v1]
pipeline = faultwrap authtoken keystonecontext ratelimit osapi_volume_app_v1

[filter:ratelimit]
paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory
limits =(POST, "*", .*, 100, MINUTE);(POST, "*/servers", ^/servers, 500, DAY);(PUT, "*", .*, 100, MINUTE);(GET, "*changes-since*", .*changes-since.*, 30, MINUTE);(DELETE, "*", .*, 100, MINUTE)

I am getting this error when I try to restart openstack-nova-api:


2013-12-05 12:51:20.035 2350 ERROR nova.wsgi [-] Ambiguous section names ['composite:openstack_compute_api_v2', 'pipeline:openstack_compute_api_v2'] for section 'openstack_compute_api_v2' (prefixed by 'app' or 'application' or 'composite' or 'composit' or 'pipeline' or 'filter-app') found in config /etc/nova/api-paste.ini
2013-12-05 12:51:20.044 2549 INFO nova.ec2.wsgi.server [-] (2549) wsgi starting up on http://0.0.0.0:8773/

2013-12-05 12:51:20.045 2350 CRITICAL nova [-] Could not load paste app 'osapi_compute' from /etc/nova/api-paste.ini
2013-12-05 12:51:20.045 2350 TRACE nova Traceback (most recent call last):
2013-12-05 12:51:20.045 2350 TRACE nova   File "/usr/bin/nova-api", line 61, in <module>
2013-12-05 12:51:20.045 2350 TRACE nova     server = service.WSGIService(api, use_ssl=should_use_ssl)
2013-12-05 12:51:20.045 2350 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/service.py", line 598, in __init__
2013-12-05 12:51:20.045 2350 TRACE nova     self.app = self.loader.load_app(name)
2013-12-05 12:51:20.045 2350 TRACE nova   File "/usr/lib/python2.6/site-packages/nova/wsgi.py", line 485, in load_app
2013-12-05 12:51:20.045 2350 TRACE nova     raise exception.PasteAppNotFound(name=name, path=self.config_path)
2013-12-05 12:51:20.045 2350 TRACE nova PasteAppNotFound: Could not load paste app 'osapi_compute' from /etc/nova/api-paste.ini
2013-12-05 12:51:20.045 2350 TRACE nova 
2013-12-05 12:51:20.407 2549 INFO nova.service [-] Parent process has died unexpectedly, exiting


Regards,

Arindam

Date: Thu, 5 Dec 2013 15:37:21 +0400
From: dmescheryakov at mirantis.com
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [savanna] This request was rate-limited. (HTTP	413)

Hello Arindam,
While deploying Hadoop cluster Savanna does pretty many API requests to Nova. Naturally, the number of requests is directly proportional to the size of the cluster.


On the other hand Nova has a protection agains users abusing API with too many requests. It is called rate limiting. You need to set limits higher than they are right now if you want to spin-up a cluster of that size. You can find details in the Nova docs:
http://docs.openstack.org/grizzly/openstack-compute/admin/content//configuring-compute-API.html


Dmitry

2013/12/5 Arindam Choudhury <arindam at live.com>




Hi,

When I try to create a big hadoop cluster (21 nodes), sometimes I am getting this error: 

2013-12-05 12:17:57.920 29553 ERROR savanna.context [-] Thread 'cluster-creating-8d093d9b-c675-4222-b53a-3319d54bc61f' fails with exception: 'This request was rate-limited. (HTTP 413)'

2013-12-05 12:17:57.920 29553 TRACE savanna.context Traceback (most recent call last):
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/savanna/context.py", line 128, in wrapper
2013-12-05 12:17:57.920 29553 TRACE savanna.context     func(*args, **kwargs)

2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/savanna/service/api.py", line 123, in _provision_cluster
2013-12-05 12:17:57.920 29553 TRACE savanna.context     i.create_cluster(cluster)

2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/savanna/service/instances.py", line 56, in create_cluster
2013-12-05 12:17:57.920 29553 TRACE savanna.context     _rollback_cluster_creation(cluster, ex)

2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-12-05 12:17:57.920 29553 TRACE savanna.context     self.gen.next()
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/savanna/service/instances.py", line 36, in create_cluster

2013-12-05 12:17:57.920 29553 TRACE savanna.context     _create_instances(cluster)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/savanna/service/instances.py", line 111, in _create_instances

2013-12-05 12:17:57.920 29553 TRACE savanna.context     _run_instance(cluster, node_group, idx, aa_groups, userdata)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/savanna/service/instances.py", line 173, in _run_instance

2013-12-05 12:17:57.920 29553 TRACE savanna.context     key_name=cluster.user_keypair_id)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/v1_1/servers.py", line 658, in create

2013-12-05 12:17:57.920 29553 TRACE savanna.context     **boot_kwargs)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/base.py", line 402, in _boot

2013-12-05 12:17:57.920 29553 TRACE savanna.context     return_raw=return_raw, **kwargs)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/base.py", line 145, in _create

2013-12-05 12:17:57.920 29553 TRACE savanna.context     _resp, body = self.api.client.post(url, body=body)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/client.py", line 232, in post

2013-12-05 12:17:57.920 29553 TRACE savanna.context     return self._cs_request(url, 'POST', **kwargs)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/client.py", line 213, in _cs_request

2013-12-05 12:17:57.920 29553 TRACE savanna.context     **kwargs)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/client.py", line 195, in _time_request

2013-12-05 12:17:57.920 29553 TRACE savanna.context     resp, body = self.request(url, method, **kwargs)
2013-12-05 12:17:57.920 29553 TRACE savanna.context   File "/root/savanna/.tox/venv/lib/python2.6/site-packages/novaclient/client.py", line 189, in request

2013-12-05 12:17:57.920 29553 TRACE savanna.context     raise exceptions.from_response(resp, body, url, method)
2013-12-05 12:17:57.920 29553 TRACE savanna.context OverLimit: This request was rate-limited. (HTTP 413)

2013-12-05 12:17:57.920 29553 TRACE savanna.context 

How I can prevent this to happen? Any help will be highly appreciated.

Regards,

Arindam
 		 	   		  

_______________________________________________

OpenStack-dev mailing list

OpenStack-dev at lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131205/5ae7357d/attachment.html>