Open Stack

I'm resurrecting this thread to provide an update on this effort.

I have been looking at Boson [1] as a base for starting developing a
service for managing quotas and reservations [2].
Boson's model would satisfy most of the requirements for this service, and
implementing additional requirements, such as hierarchical multi-tenancy
should be quite easy (at least from the high-level design perspective). In
the rest of this post I'm going to refer to this service both as "Boson"
and "quota service".

Without going into too many technical details (which I am happy to discuss
separately), the quota service will need to provide the 4 interfaces
depicted in [3].
1) The admin interface has the main purpose of keeping track of the
services which use boson, registering resources to track, and managing
their lifecycle.
2) The "quota mgmt" interface does pretty much what the "quota extension"
does for many openstack project - manage resource limits per project/users,
configure quota classes, etc.
3) The "reservation" interface handles the request/commit/cancel process
supported by many Openstack projects
4) the "usage" interface provides abstraction to access the resource usage
tracker and feed information to it.

These interfaces are then implemented by the Boson object model, depicted
in [4].
This object model is not very different from the one originally proposed
for Boson [5]
The proposed object model simplifies a bit the original one, merging the
"reservation" and "request" concepts, as well as doing without the
"SpecificResource" concept (at least for the moment). However, the proposed
object model adds the "Quota class" concept and introduces the possibility
of having child-parent relationships between Resources and User Info. The
former will allow for applying quota to resources which are scoped within a
parent resource (e.g.: static routes per logical router), whereas the
latter should enable the hierarchical multi-tenancy use case.

Keeping in mind the interfaces discussed in [3], the component diagram [6]
can be devised. There is a distinct component for each interface - plus one
component for DB management. Most of the interactions are, of course, with
the DB manager. Component design should be done in a way that the various
components are as independent as possible. There are some interactions
among components, but they can likely be replaced with interactions with
the "DB manager" component.

The Boson quota service therefore represents a centralized endpoint for
managing quotas, tracking resource usage, and performing resource
reservation.
Conceptually, this is all good; however, would it scale from an
architectural perspective? The main problem with this approach indeed is
that the quota service itself can become a huge bottleneck and add more
latency to resource processing. This problem can probably be solved looking
at the above mentioned interfaces as "roles", and expecting Boson to scale
horizontally via deployment of several instances of the service which can
implement different roles, some of which can be dedicated to specific
services.

For instance, let's consider the example architecture depicted in [7]. Here
there are 3 boson instances.
One instance has the "quota management" and "admin" role and takes care of
registering resources for enabled services, and setting/retrieving resource
limits. It might expose its own REST API endpoint over HTTP to this aim.
The other two instances have the "reservation" and "usage" roles, and
communicate with client applications over AMQP. However one instance will
listen only to "neutron" messages and the other only to "nova" messages
(nova and neutron have been used merely as an example, the same applies to
any project). This architecture will enable some sort of horizontal scale
out as every project will then have its own instance for tracking resources
and managing reservations.
However, there will still be need for some communication across the various
Boson instances. For instance in order to make a reservation, current
resource limits should be fetched interacting with the management component
or from the database. This overhead can be reduced considerably by
introducing caches, which would result particularly effective for data such
as resource limits which do not tend to change a lot over time.
Finally, it is also worth noting that in theory nothing prevents us from
running Boson's management interface as a Keystone 3rd party component and
therefore running its API on the Keystone endpoint.

Summarising, I hope I managed to convey the essentials of where I think
Boson should be place in the OpenStack ecosystem, and how would it work
from a conceptual and architectural perspective. I am aware that this is
not exhaustive at al, but is probably the best that can be achieved in a
mailing list post.

Nevertheless, there are still some fundamental questions that I am trying
to answer in a convincing way.

1) would anyone ever deploy a service aimed exclusively at managing quotas
and reservations?
This clearly makes an OpenStack deployment even more complex. The
additional complexity might outweigh the hassle of having quota management
API sparse over multiple projects and reservations performed in an
inconsistent way. Operators might an additional orchestration service for
the former, where as the latter issue might be solved either ensuring
projects do reservations in the same way, or through the library idea we
rejected (at least in terms of oslo library).

2) the reliability of a system where resource usage tracking is delegated
to an external project.
Clearly there will be many more chances of actual resource usage to differ
from the one represented in the tracker. This might easily happen in case
of communication failure, temporary unavailability of the tracker, failures
during requests processing, etc. etc.

3) The scalability of a reservation process handled within the project
versus a reservation process delegated to an external system.

I am currently adapting Boson's code to the new object model and
"synchronizing" it with the current OpenStack world - adding for instance
support for objects like 'nova objects'. However, I would like to seek your
input both on the devised architecture and most importantly on the above
questions - mostly because at the end of the day I am not entirely
convinced a service like this would actually be adopted by operators.

Regards,
Salvatore

[1] https://github.com/klmitch/boson/
[2] If your thinking of a funny name for this service, stop. If your
thinking that this is "quota as a service", stop.
[3] http://www.4shared.com/download/-NJIub0fce/boson_overview.jpg?lgfp=3000
[4] http://www.4shared.com/download/a38OINvPba/boson_obj_model.jpg?lgfp=3000
[5] https://github.com/klmitch/boson/blob/master/doc/data-model.png
[6]
http://www.4shared.com/download/3CNdnsmuce/boson_components.jpg?lgfp=3000
[7] http://www.4shared.com/download/ds0VRvNlba/boson_arch.jpg?lgfp=3000

On 28 November 2014 at 18:20, Jay Pipes <jaypipes at gmail.com> wrote:

> On 11/17/2014 05:49 PM, Salvatore Orlando wrote:
>
>> Hi all,
>>
>> I am resuming this thread following the session we had at the summit in
>> Paris (etherpad here [1])
>>
>> While there was some sort of consensus regarding what this library
>> should do, and how it should do it, the session ended with some open
>> questions which we need to address before finalising the specification.
>>
>> There was a rather large consensus that the only viable architecture
>> would be one where the quota management library owns resource usage
>> data. However, this raises further concerns around:
>> - ownership of resource usage records. As the quota library owns usage
>> data, it becomes the authoritative source of truth for it. This could be
>> problematic in some projects, particularly nova, where the resource
>> tracker currently own usage data.
>>
>
> Well, just to be clear... the resource tracker in Nova owns the usage
> records for the compute node, not the usage records for a tenant or user
> (the quota driver and DB tables own that data).
>
> Best,
> -jay
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150115/dab51636/attachment-0001.html>