Open Stack

More inline,
Salvatore

On 20 January 2016 at 16:51, Shraddha Pandhe <spandhe.openstack at gmail.com>
wrote:

> Thank you all for the comments.
>
> The client that we expect to call this API with thousands of network-ids
> is nova-scheduler.
>
> Since this call is happening in the middle of scheduling, we don't want to
> spend time in paginating or sending multiple requests. I have tens of
> thousands of networks and subnets in my test cluster right now and with
> that scale, the extension takes more than 2 seconds to return.
>

What percentage of this time is spent in the GET /v2.0/networks call?


> With multiple calls, scheduler will become very slow.
>

If the calls are serialized that is surely correct. As most production
neutron servers employ multiple workers the overhead of doing multiple
calls in parallel might however be tolerable.
I'd like to understand more about your use case. Here are some additional
questions

Is network-id the only attribute you can filter on?
Assuming Neutron provided tags in the API could you leverage those?
Why is not tenant-id a viable alternative?


>
> I agree that sending payload with GET is not recommended and most
> libraries just drop the payload for such cases.
>

Nevertheless, we're pretty much in control of that. We've already discussed
this, and doing so does not violate RFC7231, so it's ok from a protocol
perspective.
If needed, we can tweak the API request processing workflow for allowing
this.


>
>
>
> On Wed, Jan 20, 2016 at 2:27 PM, Salvatore Orlando <salv.orlando at gmail.com
> > wrote:
>
>> I tend to agree with Doug and Ryan's stance. If you need to pass 1000s of
>> network-id on a single request you're probably not doing things right on
>> the client side.
>> As Ryan suggested you can try and split the request in multiple requests
>> with acceptable URI lenght and send them in parallel; this will add some
>> overhead, but should work flawlessly.
>>
>> Once tags will be implemented you will be able to leverage those to
>> simplify your queries.
>>
>> Regarding GET requests with plenty of parameters, this discussion came up
>> on the mailing list a while ago [1]. A good proposal was made in that
>> thread but never formalised as an API-wg guideline; you consider submitting
>> a patch to the API-wg too.
>> Note however that Neutron won't be able to support it out of the box
>> considering its WSGI framework completely ignores request bodies on GET
>> requests.
>>
>> Salvatore
>>
>> [1]
>> http://lists.openstack.org/pipermail/openstack-dev/2015-November/078243.html
>>
>> On 20 January 2016 at 12:33, Ryan Brown <rybrown at redhat.com> wrote:
>>
>>> So having a URI too long error is, in this case, likely an indication
>>> that you're requesting too many things at once.
>>>
>>> You could:
>>> 1. Request 100 at a time in parallel
>>> 2. Find a query that would give you all those networks & page through
>>> the reply
>>> 3. Page through all the user's networks and filter client-side
>>>
>>> How is the user supposed to be assembling this giant UUID list? I'd
>>> think it would be easier for them to specify a query (e.g. "get usage data
>>> for all my production subnets" or something).
>>>
>>>
>>> On 01/19/2016 06:59 PM, Shraddha Pandhe wrote:
>>>
>>>> Hi folks,
>>>>
>>>>
>>>> I am writing a Neutron extension which needs to take 1000s of
>>>> network-ids as argument for filtering. The CURL call is as follows:
>>>>
>>>> curl -i -X GET
>>>> 'http://hostname:port
>>>> /neutron/v2.0/extension_name.json?net-id=fffecbd1-0f6d-4f02-aee7-ca62094830f5&net-id=fffeee07-4f94-4cff-bf8e-a2aa7be59e2e'
>>>> -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H
>>>> "X-Auth-Token: cccccccccccccccccccccccccccccccc"
>>>>
>>>>
>>>> The list of net-ids can go up to 1000s. The problem is, with such large
>>>> url, I get the "Request URI too long" error. I don't want to update this
>>>> limit as proxies can have their own limits.
>>>>
>>>> What options do I have to send 1000s of network IDs?
>>>>
>>>> 1. -d '{}' is not a recommended option for GET call and wsgi Controller
>>>> drops the data part when routing the request.
>>>>
>>>> 2. Use POST instead of GET? I will need to write the get_<resource>
>>>> logic inside create_resource logic for this to work. Its a hack, but
>>>> complies with HTTP standard.
>>>>
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>> --
>>> Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc.
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160121/ecc7ba95/attachment.html>