Open Stack

I'm not sure if this was mentioned already throughout the thread, however
as I've been working a bit on quotas in the past I might have some
additional information:

- Looking at quotas it is worth distinguishing between management (eg::
resource limits per tenant and/or users), and enforcement (eg.: can the
bakery service give me 4 cookies or did I already eat too many?)
  While for the reasons listed throughout this thread the latter should
really happen in the same context where the request is going to served,
quota management instead might be its own service, or however being done in
a common endpoint for all OpenStack resources.
- As far as quota enforcement is concerned, Dims already shared all the
relevant links. You might be already aware that we had a consensus around a
library, but hit a bit of a blocker on the fact that the library should've
introduced db model changes (at the time I devised a massive hack disguised
as abstraction around it). Considering alembic advancements (are we all
using alembic aren't we?) this should not be anymore an issue. I really
would love to have a library that does quota enforcement.
- It has also been raised a good point about securing a chunk of resources
across project, that is also related to John's point about business
quotas... I'm not sure it is necessary, but Blazar [1] kind of achieves
this - even if it was conceived with different purposes.

Salvatore

[1] https://wiki.openstack.org/wiki/Blazar


On 16 March 2016 at 18:27, John Dickinson <me at not.mn> wrote:

> There are two types of quotas you may want to enforce in an OpenStack
> project: technical and business.
>
> Technical quotas are things that are hard limits of the system based on
> either actual resources available or protecting the system itself. For
> example, you can't provision a 2TB volume if you only have 1TB of capacity
> available. Similarly, you may want to ratelimit a user to a certain number
> of operations per second in order to keep the system usable by every user.
>
> These sort of quotas should absolutely stay in the realm of each
> individual project. And, for example, if Trove needs to provision a Cinder
> volume but that fails, it's Trove's responsibility for handling that
> elegantly.
>
> Business quotas are different. This is stuff like "a user is allowed to
> provision 1TB of Cinder per Nova compute unit that is provisioned" or "a
> user can provision 1Gb of network capacity per 200TB of data stored in
> Swift". Simpler rules that don't have cross-project dependencies are
> possible too (eg "A user can have no more than 3 compute instances" or "a
> user can have no more than 100k objects or 500TB stored in Swift").
> Oftentimes, these business quotas will be tied in to (or dependent on)
> other product-specific tools like billing or CRM systems.
>
> These business quotas should have a common rules engine in an OpenStack
> deployment. I've long thought that this sort of quota enforcement is an
> authZ decision (i.e. Keystone), but perhaps it's in some other project
> (Congress?). The hard part is that if it's in a central place, that service
> has to be enormously scalable. Specifically, it has to be able to handle
> the aggregate request rate load of every service it is enforcing quotas on.
>
> If we end up with an OpenStack project that is doing centralized business
> quotas, you've got the start of building an ERP system (
> https://en.wikipedia.org/wiki/Enterprise_resource_planning). Frankly, I
> don't think we should be doing that. It's outside of our scope of building
> cloud infrastructure software.
>
> However, we should be all about fixing any problems any individual project
> has about handling technical quotas. That work should stay within its
> respective project. There's no need to consolidate or combine
> project-specific resource management because they happen to all be called
> "quotas".
>
> --John
>
>
>
>
> On 15 Mar 2016, at 23:25, Nikhil Komawar wrote:
>
> > Hello everyone,
> >
> > tl;dr;
> > I'm writing to request some feedback on whether the cross project Quotas
> > work should move ahead as a service or a library or going to a far
> > extent I'd ask should this even be in a common repository, would
> > projects prefer to implement everything from scratch in-tree? Should we
> > limit it to a guideline spec?
> >
> > But before I ask anymore, I want to specifically thank Doug Hellmann,
> > Joshua Harlow, Davanum Srinivas, Sean Dague, Sean McGinnis and  Andrew
> > Laski for the early feedback that has helped provide some good shape to
> > the already discussions.
> >
> > Some more context on what the happenings:
> > We've this in progress spec [1] up for providing context and platform
> > for such discussions. I will rephrase it to say that we plan to
> > introduce a new 'entity' in the Openstack realm that may be a library or
> > a service. Both concepts have trade-offs and the WG wanted to get more
> > ideas around such trade-offs from the larger community.
> >
> > Service:
> > This would entail creating a new project and will introduce managing
> > tables for quotas for all the projects that will use this service. For
> > example if Nova, Glance, and Cinder decide to use it, this 'entity' will
> > be responsible for handling the enforcement, management and DB upgrades
> > of the quotas logic for all resources for all three projects. This means
> > less pain for projects during the implementation and maintenance phase,
> > holistic view of the cloud and almost a guarantee of best practices
> > followed (no clutter or guessing around what different projects are
> > doing). However, it results into a big dependency; all projects rely on
> > this one service for right enforcement, avoiding races (if do not
> > incline on implementing some of that in-tree) and DB
> > migrations/upgrades. It will be at the core of the cloud and prone to
> > attack vectors, bugs and margin of error.
> >
> > Library:
> > A library could be thought of in two different ways:
> > 1) Something that does not deal with backed DB models, provides a
> > generic enforcement and management engine. To think ahead a little bit
> > it may be a ABC or even a few standard implementation vectors that can
> > be imported into a project space. The project will have it's own API for
> > quotas and the drivers will enforce different types of logic; per se
> > flat quota driver or hierarchical quota driver with custom/project
> > specific logic in project tree. Project maintains it's own DB and
> > upgrades thereof.
> > 2) A library that has models for DB tables that the project can import
> > from. Thus the individual projects will have a handy outline of what the
> > tables should look like, implicitly considering the right table values,
> > arguments, etc. Project has it's own API and implements drivers in-tree
> > by importing this semi-defined structure. Project maintains it's own
> > upgrades but will be somewhat influenced by the common repo.
> >
> > Library would keep things simple for the common repository and sourcing
> > of code can be done asynchronously as per project plans and priorities
> > without having a strong dependency. On the other hand, there is a
> > likelihood of re-implementing similar patterns in different projects
> > with individual projects taking responsibility to keep things up to
> > date. Attack vectors, bugs and margin of error are project
> responsibilities
> >
> > Third option is to avoid all of this and simply give guidelines, best
> > practices, right packages to each projects to implement quotas in-house.
> > Somewhat undesirable at this point, I'd say. But we're all ears!
> >
> > Thank you for reading and I anticipate more feedback.
> >
> > [1] https://review.openstack.org/#/c/284454/
> >
> > --
> >
> > Thanks,
> > Nikhil
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160316/077826c6/attachment.html>