Open Stack

My primary concern (and use of the numbers) was to make sure it isn’t expected on all “list” operations, as (albeit specifically for Keystone) some of the projects + backends can’t really support it. If we really are tied to SQL-isms (which is a fine approach) we can make these consistent where it can be supported.

So, tl;dr, this is fine as long as we don’t start expecting a “count” capability for all list-like-operation APIs.

—Morgan

> On Nov 20, 2014, at 8:48 AM, Sean Dague <sean at dague.net> wrote:
> 
> I'm looking at the Nova spec, and it seems very taylored to a specific
> GUI. I'm also not sure that 17128 errors is more useful than 500+ errors
> when presenting to the user (the following in my twitter stream made me
> think about that this morning -
> https://twitter.com/NINK/status/535299029383380992)
> 
> 500+ also better describes the significant figures we're talking about here.
> 
> 	-Sean
> 
> On 11/20/2014 11:28 AM, Morgan Fainberg wrote:
>> The only thing I want to caution against is making a SQL-specific
>> choice. In the case of some other backends, it may not be possible (for
>> an extremely large dataset) to get a full count, where SQL does this
>> fairly elegantly. For example, LDAP (in some cases) may have an
>> administrative limit that will say that no more than 10,000 entries
>> would be returned; likely you’re going to have an issue, since you need
>> to issue the query and see how many things match, if you hit the overall
>> limit you’ll get the same count every time (but possibly a different
>> dataset).
>> 
>> I want to be very careful that we’re not recommending functionality as a
>> baseline that should be used as a pattern across all similar APIs,
>> especially since we have some backends/storage systems that can’t
>> elegantly always support it.
>> 
>> Personally, I like Gerrit’s model (as Sean described) - with the above
>> caveat that not all backends support this type of count.
>> 
>> Cheers,
>> Morgan
>> 
>>> On Nov 20, 2014, at 8:04 AM, Salvatore Orlando <sorlando at nicira.com
>>> <mailto:sorlando at nicira.com>> wrote:
>>> 
>>> The Nova proposal appears to be identical to neutron's, at least from
>>> a consumer perspective.
>>> 
>>> If I were to pick a winner, I'd follow Sean's advice regarding the
>>> 'more' attribute in responses, and put the total number of resources
>>> there; I would also take Jay's advice of including the total only if
>>> requested with a query param. In this way a user can retrieve the
>>> total number of items regardless of the current pagination index (in
>>> my first post I suggested the total number should be returned only on
>>> the first page of results).
>>> 
>>> Therefore one could ask for a total number of resources with something
>>> like the following:
>>> 
>>> GET /some_resources?include_total=1
>>> 
>>> and obtain a response like the following:
>>> 
>>> {'resources': [{meh}, {meh}, {meh_again}],
>>>  'something': {
>>>       '_links': {'prev': ..., 'next': ...},
>>>       'total': agazillion}
>>> }
>>> 
>>> where the exact structure and naming of 'something' depends on the
>>> outcome of the discussion at [1]
>>> 
>>> Salvatore
>>> 
>>> [1] https://review.openstack.org/#/c/133660/7/guidelines/representation_structure.rst
>>> 
>>> On 20 November 2014 15:24, Christopher Yeoh <cbkyeoh at gmail.com
>>> <mailto:cbkyeoh at gmail.com>> wrote:
>>> 
>>>    On Thu, 20 Nov 2014 14:47:16 +0100
>>>    Salvatore Orlando <sorlando at nicira.com
>>>    <mailto:sorlando at nicira.com>> wrote:
>>> 
>>>> Aloha guardians of the API!
>>>> 
>>>> I haven recently* reviewed a spec for neutron [1] proposing a
>>>> distinct URI for returning resource count on list operations.
>>>> This proposal is for selected neutron resources, but I believe the
>>>> topic is general enough to require a guideline for the API working
>>>> group. Your advice is therefore extremely valuable.
>>>> 
>>>> In a nutshell the proposal is to retrieve resource count in the
>>>> following way:
>>>> GET /<prefix>/<resource_name>/count
>>>> 
>>>> In my limited experience with RESTful APIs, I've never encountered
>>>> one that does counting in this way. This obviously does not mean
>>>    it's
>>>> a bad idea. I think it's not great from a usability perspective to
>>>> require two distinct URIs to fetch the first page and then the total
>>>> number of elements. I reckon the first response page for a list
>>>> operation might include also the total count. For example:
>>>> 
>>>> {'resources': [{meh}, {meh}, {meh_again}],
>>>> 'resource_count': 55
>>>> <link_to_next_page>}
>>>> 
>>>> I am however completely open to consider other alternatives.
>>>> What is your opinion on this matter?
>>> 
>>>    FWIW there is a nova spec proposed for counting resources as
>>>    well (I think it might have been previously approved for Juno).
>>> 
>>>    https://review.openstack.org/#/c/134279/
>>> 
>>>    I haven't compared the two, but I can't think of a reason we'd
>>>    need to be any different between projects here.
>>> 
>>>    Regards,
>>> 
>>>    Chris
>>> 
>>>> 
>>>> Regards,
>>>> Salvatore
>>>> 
>>>> 
>>>> * it's been 10 days now
>>>> 
>>>> [1] https://review.openstack.org/#/c/102199/
>>> 
>>> 
>>>    _______________________________________________
>>>    OpenStack-dev mailing list
>>>    OpenStack-dev at lists.openstack.org
>>>    <mailto:OpenStack-dev at lists.openstack.org>
>>>    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> <mailto:OpenStack-dev at lists.openstack.org>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> 
> -- 
> Sean Dague
> http://dague.net
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev