Open Stack

Hello Telemetry Team,

I see that the Panko API does not provide a way to go through all events in
the order they were stored in database. Panko sorts events by generated and
message_id fields and uses message_id field as a marker for pagination. So,
the following case is possible:

Before the first request we had the following events in database:
+-------------------+------------------------------+
|| message_id1 | 2017-08-29T00:05.00 ||
+-------------------+------------------------------+
|| message_id2 | 2017-08-29T00:06.00 ||
+-------------------+------------------------------+
|| message_id3 | 2017-08-29T00:08.00 ||
+-------------------+------------------------------+

During the first request we got all three events:
GET /v2/events?limit=3

After the request some new events were added:
+-------------------+------------------------------+
|| message_id1 | 2017-08-29T00:05.00 ||
+-------------------+------------------------------+
|| message_id2 | 2017-08-29T00:06.00 ||
+-------------------+------------------------------+
|| message_id4 | 2017-08-29T00:07.00 ||
+-------------------+------------------------------+
|| message_id3 | 2017-08-29T00:08.00 ||
+-------------------+------------------------------+
|| message_id5 | 2017-08-29T00:09.00 ||
+-------------------+------------------------------+
|| message_id6 | 2017-08-29T00:09.00 ||
+-------------------+------------------------------+

To proceed next we set marker to message_id3:
GET /v2/events?marker=message_id3&limit=3

In this case we will get only two new events (message_id5, message_id6). To
get message_id4 we need to make the first request again.

That all makes difficult such scenarios when API clients need to process
events as they appear in database.

I'd like to discuss how we can extend the current API to provide a way to
go through all events in the order they were stored in database.

It looks like cursor API might be useful in this case. We can introduce new
"cursor" query parameter. It will work as a marker for pagination, but will
determine "special" sorting of events. In this case the request might look
like:
GET /v2/events?cursor=-1&limit=100

The response will return a list of events and next cursor:
{
    events: []
    next_cursor: 12345
}

So, a request to get the next portion of events will look like:
GET /v2/events?cursor=12345&limit=100

I will be glad to hear your comments, suggestions.

Thanks,
Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170829/8dee1ce9/attachment.html>