Open Stack

On 12/16/2013 03:54 PM, Ildikó Váncsa wrote:
> Hi guys,
>
> The first working version of the Complex filter expressions in API
> queries blueprint [1] was pushed for review[2].
>
> We implemented a new query REST resource in order to provide rich query
> functionality for samples, alarms and alarm history. The future plans
> (in separated blueprints) with this new functionality is extending it to
> support Statistics and stored queries. The new feature is documented on
> Launchpad wiki[3], with an example for how to use the new query on the API.
>
> What is your opinion about this solution?
>
> I would appreciate some review comments and/or feedback on the
> implementation. :)

Hi Ildiko, thanks for your proposed API for complex querying in 
Ceilometer. Unfortunately, I'm not a fan of the approach taken, but I do 
see some definite need/use cases here.

My main objection to the proposed solution is that it violates the 
principle in all of the OpenStack REST APIs that a POST request 
*creates* a resource. In the proposed API, you use:

POST /query/$resource

to actually retrieve records of type $resource. In all the other 
OpenStack REST APIs, the above request would create a $resource 
subresource of a "query" resource. And, to be honest, people expect HTTP 
REST APIs to use the GET HTTP method for querying, not POST. It's an 
anti-pattern to use POST in this way.

Now, that said... I think that the advanced query interface you propose 
does indeed have real-world, demanded use cases. Right now, you are 100% 
correct that the existing GET request filters are simplistic and don't 
support either aggregation or advanced union or intersection queries.

I would definitely be supportive of using POST to store "saved queries" 
(as you mention in your wiki page). However, the main query interface 
should remain the GET HTTP method, including for ad-hoc advanced querying.

So, what I would like to see is essentially a removal of the "query" 
resource, and instead tack on your advanced Ceilometer domain-specific 
language to the supported GET query arguments. This would have two 
advantages:

1) You will not need to re-implement the orderby and limit expressions. 
Virtually all other OpenStack APIs (including Ceilometer) use the 
"limit" and "sort_by" query parameters already, so those should be used 
as-is.

2) Users will already be familiar with the standard GET /samples, GET 
/alarms, etc query interface, and all they would need to learn is how to 
encode the advanced query parameters properly. No need to 
learn/implement new resource endpoints.

You used this English-language example of an advanced query:

"Check for cpu_util samples reported between 18:00-18:15 or between 
18:30 - 18:45 where the utilization is between 23 and 26 percent."

and had the following POST request JSON-ified body:

POST /query/meters
["and",
   ["and",
     ["and",
       ["=", "counter_name", "cpu_util"],
       [">", "counter_volume", 0.23]],
     ["and",
       ["=", "counter_name", "cpu_util"],
       ["<", "counter_volume", 0.26]]],
   ["or",
     ["and",
	  [">", "timestamp", "2013-12-01T18:00:00"],
	  ["<", "timestamp", "2013-12-01T18:15:00"]],
     ["and",
	  [">", "timestamp", "2013-12-01T18:30:00"],
	  ["<", "timestamp", "2013-12-01T18:45:00"]]]]

(note that the above doesn't actually correspond to the English-language 
query... you would not want the third-level "and"s and you would want >= 
and <= for the temporal BETWEEN clause...)

The equivalent GET request might be encoded like so:

GET 
/meters?expr=(expr1%20or%20expr2)%20and%20expr3&expr1=(timestamp%3E%3D2013-12-01T18%3A00%3A00%20and%20timestamp%3C%3D2013-12-01T18%3A15%3A00)&expr2=(timestamp%3E%3D2013-12-01T18%3A30%3A00%20and%20timestamp%3C%3D2013-12-01T18%3A45%3A00)&expr3=(counter_name%3D%27cpu_util%27%20and%20(counter_volume%20%3E%200.23%20and%20counter_volume%20%3C%200.26))

Which is just the following, with the values url-encoded:

expr = (expr1 or expr2) and expr3
expr1 = (timestamp>=2013-12-01T18:00:00 and timestamp<=2013-12-01T18:15:00 )
expr2 = (timestamp>=2013-12-01T18:30:00 and timestamp<=2013-12-01T18:45:00)
expr3 = (counter_name='cpu_util' and (counter_volume>0.23 and 
counter_volume<0.26))

I know the expression might not look as nice as POST /query/meters, but 
it is in-line with Internet custom.

I would definitely support the use of your POST with JSON-encoded query 
DSL for stored views, though. For example, to save a stored query for 
the above report:

POST /reports
["and",
   ["and",
       ["=", "counter_name", "cpu_util"],
       ["between", "counter_volume", 0.23, 0.26],
   ["or",
     ["and",
	  [">", "timestamp", "2013-12-01T18:00:00"],
	  ["<", "timestamp", "2013-12-01T18:15:00"]],
     ["and",
	  [">", "timestamp", "2013-12-01T18:30:00"],
	  ["<", "timestamp", "2013-12-01T18:45:00"]]]]

And then you could issue the same query using the GET of the returned 
report UUID or report name...

GET /reports/$REPORT_ID

Best,
-jay

> [1]
> https://blueprints.launchpad.net/ceilometer/+spec/complex-filter-expressions-in-api-queries
>
>
> [2]
> https://review.openstack.org/#/q/status:open+project:openstack/ceilometer+branch:master+topic:bp/complex-filter-expressions-in-api-queries,n,z
>
>
> [3]
> https://wiki.openstack.org/wiki/Ceilometer/ComplexFilterExpressionsInAPIQueries
>
> Thanks and Best Regards,
>
> Ildiko
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>