[openstack-dev] [neutron][api] GET call with huge argument list

Shraddha Pandhe spandhe.openstack at gmail.com
Thu Jan 21 00:51:02 UTC 2016


Thank you all for the comments.

The client that we expect to call this API with thousands of network-ids is
nova-scheduler.

Since this call is happening in the middle of scheduling, we don't want to
spend time in paginating or sending multiple requests. I have tens of
thousands of networks and subnets in my test cluster right now and with
that scale, the extension takes more than 2 seconds to return. With
multiple calls, scheduler will become very slow.

I agree that sending payload with GET is not recommended and most libraries
just drop the payload for such cases.



On Wed, Jan 20, 2016 at 2:27 PM, Salvatore Orlando <salv.orlando at gmail.com>
wrote:

> I tend to agree with Doug and Ryan's stance. If you need to pass 1000s of
> network-id on a single request you're probably not doing things right on
> the client side.
> As Ryan suggested you can try and split the request in multiple requests
> with acceptable URI lenght and send them in parallel; this will add some
> overhead, but should work flawlessly.
>
> Once tags will be implemented you will be able to leverage those to
> simplify your queries.
>
> Regarding GET requests with plenty of parameters, this discussion came up
> on the mailing list a while ago [1]. A good proposal was made in that
> thread but never formalised as an API-wg guideline; you consider submitting
> a patch to the API-wg too.
> Note however that Neutron won't be able to support it out of the box
> considering its WSGI framework completely ignores request bodies on GET
> requests.
>
> Salvatore
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2015-November/078243.html
>
> On 20 January 2016 at 12:33, Ryan Brown <rybrown at redhat.com> wrote:
>
>> So having a URI too long error is, in this case, likely an indication
>> that you're requesting too many things at once.
>>
>> You could:
>> 1. Request 100 at a time in parallel
>> 2. Find a query that would give you all those networks & page through the
>> reply
>> 3. Page through all the user's networks and filter client-side
>>
>> How is the user supposed to be assembling this giant UUID list? I'd think
>> it would be easier for them to specify a query (e.g. "get usage data for
>> all my production subnets" or something).
>>
>>
>> On 01/19/2016 06:59 PM, Shraddha Pandhe wrote:
>>
>>> Hi folks,
>>>
>>>
>>> I am writing a Neutron extension which needs to take 1000s of
>>> network-ids as argument for filtering. The CURL call is as follows:
>>>
>>> curl -i -X GET
>>> 'http://hostname:port
>>> /neutron/v2.0/extension_name.json?net-id=fffecbd1-0f6d-4f02-aee7-ca62094830f5&net-id=fffeee07-4f94-4cff-bf8e-a2aa7be59e2e'
>>> -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H
>>> "X-Auth-Token: cccccccccccccccccccccccccccccccc"
>>>
>>>
>>> The list of net-ids can go up to 1000s. The problem is, with such large
>>> url, I get the "Request URI too long" error. I don't want to update this
>>> limit as proxies can have their own limits.
>>>
>>> What options do I have to send 1000s of network IDs?
>>>
>>> 1. -d '{}' is not a recommended option for GET call and wsgi Controller
>>> drops the data part when routing the request.
>>>
>>> 2. Use POST instead of GET? I will need to write the get_<resource>
>>> logic inside create_resource logic for this to work. Its a hack, but
>>> complies with HTTP standard.
>>>
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>> --
>> Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc.
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160120/34d53bd1/attachment.html>


More information about the OpenStack-dev mailing list