[openstack-dev] [api] Counting resources

Sean Dague sean at dague.net
Thu Nov 20 16:48:05 UTC 2014


I'm looking at the Nova spec, and it seems very taylored to a specific
GUI. I'm also not sure that 17128 errors is more useful than 500+ errors
when presenting to the user (the following in my twitter stream made me
think about that this morning -
https://twitter.com/NINK/status/535299029383380992)

500+ also better describes the significant figures we're talking about here.

	-Sean

On 11/20/2014 11:28 AM, Morgan Fainberg wrote:
> The only thing I want to caution against is making a SQL-specific
> choice. In the case of some other backends, it may not be possible (for
> an extremely large dataset) to get a full count, where SQL does this
> fairly elegantly. For example, LDAP (in some cases) may have an
> administrative limit that will say that no more than 10,000 entries
> would be returned; likely you’re going to have an issue, since you need
> to issue the query and see how many things match, if you hit the overall
> limit you’ll get the same count every time (but possibly a different
> dataset).
> 
> I want to be very careful that we’re not recommending functionality as a
> baseline that should be used as a pattern across all similar APIs,
> especially since we have some backends/storage systems that can’t
> elegantly always support it.
> 
> Personally, I like Gerrit’s model (as Sean described) - with the above
> caveat that not all backends support this type of count.
> 
> Cheers,
> Morgan
> 
>> On Nov 20, 2014, at 8:04 AM, Salvatore Orlando <sorlando at nicira.com
>> <mailto:sorlando at nicira.com>> wrote:
>>
>> The Nova proposal appears to be identical to neutron's, at least from
>> a consumer perspective.
>>
>> If I were to pick a winner, I'd follow Sean's advice regarding the
>> 'more' attribute in responses, and put the total number of resources
>> there; I would also take Jay's advice of including the total only if
>> requested with a query param. In this way a user can retrieve the
>> total number of items regardless of the current pagination index (in
>> my first post I suggested the total number should be returned only on
>> the first page of results).
>>
>> Therefore one could ask for a total number of resources with something
>> like the following:
>>
>> GET /some_resources?include_total=1
>>
>> and obtain a response like the following:
>>
>> {'resources': [{meh}, {meh}, {meh_again}],
>>   'something': {
>>        '_links': {'prev': ..., 'next': ...},
>>        'total': agazillion}
>>  }
>>
>>  where the exact structure and naming of 'something' depends on the
>> outcome of the discussion at [1]
>>
>> Salvatore
>>
>> [1] https://review.openstack.org/#/c/133660/7/guidelines/representation_structure.rst
>>
>> On 20 November 2014 15:24, Christopher Yeoh <cbkyeoh at gmail.com
>> <mailto:cbkyeoh at gmail.com>> wrote:
>>
>>     On Thu, 20 Nov 2014 14:47:16 +0100
>>     Salvatore Orlando <sorlando at nicira.com
>>     <mailto:sorlando at nicira.com>> wrote:
>>
>>     > Aloha guardians of the API!
>>     >
>>     > I haven recently* reviewed a spec for neutron [1] proposing a
>>     > distinct URI for returning resource count on list operations.
>>     > This proposal is for selected neutron resources, but I believe the
>>     > topic is general enough to require a guideline for the API working
>>     > group. Your advice is therefore extremely valuable.
>>     >
>>     > In a nutshell the proposal is to retrieve resource count in the
>>     > following way:
>>     > GET /<prefix>/<resource_name>/count
>>     >
>>     > In my limited experience with RESTful APIs, I've never encountered
>>     > one that does counting in this way. This obviously does not mean
>>     it's
>>     > a bad idea. I think it's not great from a usability perspective to
>>     > require two distinct URIs to fetch the first page and then the total
>>     > number of elements. I reckon the first response page for a list
>>     > operation might include also the total count. For example:
>>     >
>>     > {'resources': [{meh}, {meh}, {meh_again}],
>>     >  'resource_count': 55
>>     >  <link_to_next_page>}
>>     >
>>     > I am however completely open to consider other alternatives.
>>     > What is your opinion on this matter?
>>
>>     FWIW there is a nova spec proposed for counting resources as
>>     well (I think it might have been previously approved for Juno).
>>
>>     https://review.openstack.org/#/c/134279/
>>
>>     I haven't compared the two, but I can't think of a reason we'd
>>     need to be any different between projects here.
>>
>>     Regards,
>>
>>     Chris
>>
>>     >
>>     > Regards,
>>     > Salvatore
>>     >
>>     >
>>     > * it's been 10 days now
>>     >
>>     > [1] https://review.openstack.org/#/c/102199/
>>
>>
>>     _______________________________________________
>>     OpenStack-dev mailing list
>>     OpenStack-dev at lists.openstack.org
>>     <mailto:OpenStack-dev at lists.openstack.org>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> <mailto:OpenStack-dev at lists.openstack.org>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Sean Dague
http://dague.net



More information about the OpenStack-dev mailing list