Open Stack

Hi Amrith,

Thank you for your feedback and asking all the important questions. Over
the last week, I've interacted with people who are willing to work on
this, who have been involved in this, who have stake in this, etc. Based
on the digest of all of those I intend to reply to your questions.
Again, this initiative is still evolving and gladly we've a cross
project session proposed for the Austin summit [1].

Please find my responses inline.

[1] https://etherpad.openstack.org/p/newton-cross-project-sessions

On 3/16/16 8:27 AM, Amrith Kumar wrote:
> Nikhil, thank you for the very timely posting. This is a topic that has
> been discussed quite a bit recently within the Trove team. I've read the
> document you reference as [1] and I have not been part of earlier
> conversations on this subject so I may be missing some context here.
>
> I feel that the conversation (in [1], in the email thread) has gone to a
> discussion of implementation details (library vs. service, quota
> enforcement engine, interface, ...) when I think there is still some
> ambiguity in my mind about the requirements. What is it that this
> capability will provide and what is the contract implied when a service
> adopts this model.
>

I had raised two questions 1) if we needed the specifics as a part of
the cross project spec (in general) and 2) how do these implementations
get there. I think the intent is to build on those details and not
necessarily provide a authority over them. However, the biggest voices
are saying it's a hard code to get right. So, the spec will need to
updated to reflect accordingly -- whichever has been the successful
attempt of getting it right.

> For example, consider this case that we see in Trove. In response to a
> user request to create a cluster of databases, Trove must provision
> storage (cinder), compute (nova), networks (let's say neutron), and so
> on. As stated by Boris in his email, it would be ideal if Trove had a
> confirmation from all projects that there was quota available for the
> requests that would be made before the requests actually are made. This
> implies therefore that participating projects (cinder, nova, neutron,
> ...) would have to support some reservations scheme and subsequently
> honor requests based on a reservation. So, I think there's more to this
> than just another library or project, there's an implication for
> projects that wish to participate in this scheme. Or am I wrong in this
> understanding?

I think the eventual plan is to get to this state. But building
everything from scratch will mean getting things right first and then
incrementally improving!

> Several have observed that we have a 'transaction capable DB to help
> us'. Let me go further and say that we have a distributed transaction
> capable DB (yes, if you squint just the right way, MySQL can do 2phase
> commit) and we should leverage that to the hilt.
>
> But, I'd like to get caught up on the requirements that this capability
> is to provide before we get into implementations, so if there are any
> descriptions of those requirements, I'd appreciate getting pointers to that.
>

Yes, the PWG is helping build a user story around the requirements. But
those will be more on the conceptual level and probably what you are
looking for to ensure that it accommodates yours?

I would love for you to be involved in the discussions and it would be
nice if you can join the Quota WG standup (
http://eavesdrop.openstack.org/#Cross-project_Quotas_and_Nested_Quotas_Working_Group_Virtual_Standup
) once in a while on Mondays to bring in your perspective.

> Thanks,
>
> -amrith
>
>
> [1] https://review.openstack.org/#/c/284454/
>
> --
> Amrith Kumar, CTO                   | amrith at tesora.com
> Tesora, Inc                         | @amrithkumar
> 125 CambridgePark Drive, Suite 400  | http://www.tesora.com
> Cambridge, MA. 02140                | GPG: 0x5e48849a9d21a29b 
>
> On 03/16/2016 02:25 AM, Nikhil Komawar wrote:
>> Hello everyone,
>>
>> tl;dr;
>> I'm writing to request some feedback on whether the cross project Quotas
>> work should move ahead as a service or a library or going to a far
>> extent I'd ask should this even be in a common repository, would
>> projects prefer to implement everything from scratch in-tree? Should we
>> limit it to a guideline spec?
>>
>> But before I ask anymore, I want to specifically thank Doug Hellmann,
>> Joshua Harlow, Davanum Srinivas, Sean Dague, Sean McGinnis and  Andrew
>> Laski for the early feedback that has helped provide some good shape to
>> the already discussions.
>>
>> Some more context on what the happenings:
>> We've this in progress spec [1] up for providing context and platform
>> for such discussions. I will rephrase it to say that we plan to
>> introduce a new 'entity' in the Openstack realm that may be a library or
>> a service. Both concepts have trade-offs and the WG wanted to get more
>> ideas around such trade-offs from the larger community.
>>
>> Service:
>> This would entail creating a new project and will introduce managing
>> tables for quotas for all the projects that will use this service. For
>> example if Nova, Glance, and Cinder decide to use it, this 'entity' will
>> be responsible for handling the enforcement, management and DB upgrades
>> of the quotas logic for all resources for all three projects. This means
>> less pain for projects during the implementation and maintenance phase,
>> holistic view of the cloud and almost a guarantee of best practices
>> followed (no clutter or guessing around what different projects are
>> doing). However, it results into a big dependency; all projects rely on
>> this one service for right enforcement, avoiding races (if do not
>> incline on implementing some of that in-tree) and DB
>> migrations/upgrades. It will be at the core of the cloud and prone to
>> attack vectors, bugs and margin of error.
>>
>> Library:
>> A library could be thought of in two different ways:
>> 1) Something that does not deal with backed DB models, provides a
>> generic enforcement and management engine. To think ahead a little bit
>> it may be a ABC or even a few standard implementation vectors that can
>> be imported into a project space. The project will have it's own API for
>> quotas and the drivers will enforce different types of logic; per se
>> flat quota driver or hierarchical quota driver with custom/project
>> specific logic in project tree. Project maintains it's own DB and
>> upgrades thereof.
>> 2) A library that has models for DB tables that the project can import
>> from. Thus the individual projects will have a handy outline of what the
>> tables should look like, implicitly considering the right table values,
>> arguments, etc. Project has it's own API and implements drivers in-tree
>> by importing this semi-defined structure. Project maintains it's own
>> upgrades but will be somewhat influenced by the common repo.
>>
>> Library would keep things simple for the common repository and sourcing
>> of code can be done asynchronously as per project plans and priorities
>> without having a strong dependency. On the other hand, there is a
>> likelihood of re-implementing similar patterns in different projects
>> with individual projects taking responsibility to keep things up to
>> date. Attack vectors, bugs and margin of error are project responsibilities
>>
>> Third option is to avoid all of this and simply give guidelines, best
>> practices, right packages to each projects to implement quotas in-house.
>> Somewhat undesirable at this point, I'd say. But we're all ears!
>>
>> Thank you for reading and I anticipate more feedback.
>>
>> [1] https://review.openstack.org/#/c/284454/
>>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Thanks,
Nikhil