Open Stack

On Oct 8, 2014, at 7:03 AM, Davanum Srinivas <davanum at gmail.com> wrote:

> Salvatore, Joe,
> 
> We do have this at the moment:
> 
> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/quota.py
> 
> — dims

If someone wants to drive creating a useful library during kilo, please consider adding the topic to the etherpad we’re using to plan summit sessions and then come participate in the Oslo meeting this Friday 16:00 UTC.

https://etherpad.openstack.org/p/kilo-oslo-summit-topics

Doug

> 
> On Wed, Oct 8, 2014 at 2:29 AM, Salvatore Orlando <sorlando at nicira.com> wrote:
>> 
>> On 8 October 2014 04:13, Joe Gordon <joe.gordon0 at gmail.com> wrote:
>>> 
>>> 
>>> On Fri, Oct 3, 2014 at 10:47 AM, Morgan Fainberg
>>> <morgan.fainberg at gmail.com> wrote:
>>>> 
>>>> Keeping the enforcement local (same way policy works today) helps limit
>>>> the fragility, big +1 there.
>>>> 
>>>> I also agree with Vish, we need a uniform way to talk about quota
>>>> enforcement similar to how we have a uniform policy language / enforcement
>>>> model (yes I know it's not perfect, but it's far closer to uniform than
>>>> quota management is).
>>> 
>>> 
>>> It sounds like maybe we should have an oslo library for quotas? Somewhere
>>> where we can share the code,but keep the operations local to each service.
>> 
>> 
>> This is what I had in mind as well. A simple library for quota enforcement
>> which can be used regardless of where and how you do it, which might depend
>> on the application business logic, the WSGI framework in use, or other
>> factors.
>> 
>>> 
>>>> 
>>>> 
>>>> If there is still interest of placing quota in keystone, let's talk about
>>>> how that will work and what will be needed from Keystone . The previous
>>>> attempt didn't get much traction and stalled out early in implementation. If
>>>> we want to revisit this lets make sure we have the resources needed and
>>>> spec(s) in progress / info on etherpads (similar to how the multitenancy
>>>> stuff was handled at the last summit) as early as possible.
>>> 
>>> 
>>> Why not centralize quota management via the python-openstackclient, what
>>> is the benefit of getting keystone involved?
>> 
>> 
>> Providing this through the openstack client in my opinion has the
>> disadvantage that users which either use the REST API direct or write their
>> own clients won't leverage it. I don't think it's a reasonable assumption
>> that everybody will use python-openstackclient, is it?
>> 
>> Said that, storing quotas in keystone poses a further challenge to the
>> scalability of the system, which we shall perhaps address by using
>> appropriate caching strategies and leveraging keystone notifications. Until
>> we get that, I think that the openstack client will be the best way of
>> getting a unified quota management experience.
>> 
>> Salvatore
>> 
>> 
>>>> 
>>>> Cheers,
>>>> Morgan
>>>> 
>>>> Sent via mobile
>>>> 
>>>> 
>>>> On Friday, October 3, 2014, Salvatore Orlando <sorlando at nicira.com>
>>>> wrote:
>>>>> 
>>>>> Thanks Vish,
>>>>> 
>>>>> this seems a very reasonable first step as well - and since most
>>>>> projects would be enforcing quotas in the same way, the shared library would
>>>>> be the logical next step.
>>>>> After all this is quite the same thing we do with authZ.
>>>>> 
>>>>> Duncan is expressing valid concerns which in my opinion can be addressed
>>>>> with an appropriate design - and a decent implementation.
>>>>> 
>>>>> Salvatore
>>>>> 
>>>>> On 3 October 2014 18:25, Vishvananda Ishaya <vishvananda at gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>> The proposal in the past was to keep quota enforcement local, but to
>>>>>> put the resource limits into keystone. This seems like an obvious first
>>>>>> step to me. Then a shared library for enforcing quotas with decent
>>>>>> performance should be next. The quota calls in nova are extremely
>>>>>> inefficient right now and it will only get worse when we try to add
>>>>>> hierarchical projects and quotas.
>>>>>> 
>>>>>> Vish
>>>>>> 
>>>>>> On Oct 3, 2014, at 7:53 AM, Duncan Thomas <duncan.thomas at gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Taking quota out of the service / adding remote calls for quota
>>>>>>> management is going to make things fragile - you've somehow got to
>>>>>>> deal with the cases where your quota manager is slow, goes away,
>>>>>>> hiccups, drops connections etc. You'll also need some way of
>>>>>>> reconciling actual usage against quota usage periodically, to detect
>>>>>>> problems.
>>>>>>> 
>>>>>>> On 3 October 2014 15:03, Salvatore Orlando <sorlando at nicira.com>
>>>>>>> wrote:
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> Quota management is currently one of those things where every
>>>>>>>> openstack
>>>>>>>> project does its own thing. While quotas are obviously managed in a
>>>>>>>> similar
>>>>>>>> way for each project, there are subtle differences which ultimately
>>>>>>>> result
>>>>>>>> in lack of usability.
>>>>>>>> 
>>>>>>>> I recall that in the past there have been several calls for unifying
>>>>>>>> quota
>>>>>>>> management. The blueprint [1] for instance, hints at the possibility
>>>>>>>> of
>>>>>>>> storing quotas in keystone.
>>>>>>>> On the other hand, the blazar project [2, 3] seems to aim at solving
>>>>>>>> this
>>>>>>>> problem for good enabling resource reservation and therefore
>>>>>>>> potentially
>>>>>>>> freeing openstack projects from managing and enforcing quotas.
>>>>>>>> 
>>>>>>>> While Blazar is definetely a good thing to have, I'm not entirely
>>>>>>>> sure we
>>>>>>>> want to make it a "required" component for every deployment. Perhaps
>>>>>>>> single
>>>>>>>> projects should still be able to enforce quota. On the other hand,
>>>>>>>> at least
>>>>>>>> on paper, the idea of making Keystone "THE" endpoint for managing
>>>>>>>> quotas,
>>>>>>>> and then letting the various project enforce them, sounds promising
>>>>>>>> - is
>>>>>>>> there any reason for which this blueprint is stalled to the point
>>>>>>>> that it
>>>>>>>> seems forgotten now?
>>>>>>>> 
>>>>>>>> I'm coming to the mailing list with these random questions about
>>>>>>>> quota
>>>>>>>> management, for two reasons:
>>>>>>>> 1) despite developing and using openstack on a daily basis I'm still
>>>>>>>> confused by quotas
>>>>>>>> 2) I've found a race condition in neutron quotas and the fix is not
>>>>>>>> trivial.
>>>>>>>> So, rather than start coding right away, it might probably make more
>>>>>>>> sense
>>>>>>>> to ask the community if there is already a known better approach to
>>>>>>>> quota
>>>>>>>> management - and obviously enforcement.
>>>>>>>> 
>>>>>>>> Thanks in advance,
>>>>>>>> Salvatore
>>>>>>>> 
>>>>>>>> [1] https://blueprints.launchpad.net/keystone/+spec/service-metadata
>>>>>>>> [2] https://wiki.openstack.org/wiki/Blazar
>>>>>>>> [3] https://review.openstack.org/#/q/project:stackforge/blazar,n,z
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> OpenStack-dev mailing list
>>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Duncan Thomas
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>> 
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> 
> 
> -- 
> Davanum Srinivas :: https://twitter.com/dims
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev