Open Stack

Hi Doug,

do you know if the existing quota oslo-incubator module has already some
active consumers?
In the meanwhile I've pushed a spec to neutron-specs for improving quota
management there [1]

Now, I can either work on the oslo-incubator module and leverage it in
Neutron, or develop the quota module in Neutron, and move it to
oslo-incubator once we validate it with Neutron. The latter approach seems
easier from a workflow perspective - as it avoid the intermediate steps of
moving code from oslo-incubator to neutron. On the other hand it will delay
adoption in oslo-incubator.

What's your opinion?

Regards,
Salvatore

[1] https://review.openstack.org/#/c/128318/

On 8 October 2014 18:52, Doug Hellmann <doug at doughellmann.com> wrote:

>
> On Oct 8, 2014, at 7:03 AM, Davanum Srinivas <davanum at gmail.com> wrote:
>
> > Salvatore, Joe,
> >
> > We do have this at the moment:
> >
> >
> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/quota.py
> >
> > — dims
>
> If someone wants to drive creating a useful library during kilo, please
> consider adding the topic to the etherpad we’re using to plan summit
> sessions and then come participate in the Oslo meeting this Friday 16:00
> UTC.
>
> https://etherpad.openstack.org/p/kilo-oslo-summit-topics
>
> Doug
>
> >
> > On Wed, Oct 8, 2014 at 2:29 AM, Salvatore Orlando <sorlando at nicira.com>
> wrote:
> >>
> >> On 8 October 2014 04:13, Joe Gordon <joe.gordon0 at gmail.com> wrote:
> >>>
> >>>
> >>> On Fri, Oct 3, 2014 at 10:47 AM, Morgan Fainberg
> >>> <morgan.fainberg at gmail.com> wrote:
> >>>>
> >>>> Keeping the enforcement local (same way policy works today) helps
> limit
> >>>> the fragility, big +1 there.
> >>>>
> >>>> I also agree with Vish, we need a uniform way to talk about quota
> >>>> enforcement similar to how we have a uniform policy language /
> enforcement
> >>>> model (yes I know it's not perfect, but it's far closer to uniform
> than
> >>>> quota management is).
> >>>
> >>>
> >>> It sounds like maybe we should have an oslo library for quotas?
> Somewhere
> >>> where we can share the code,but keep the operations local to each
> service.
> >>
> >>
> >> This is what I had in mind as well. A simple library for quota
> enforcement
> >> which can be used regardless of where and how you do it, which might
> depend
> >> on the application business logic, the WSGI framework in use, or other
> >> factors.
> >>
> >>>
> >>>>
> >>>>
> >>>> If there is still interest of placing quota in keystone, let's talk
> about
> >>>> how that will work and what will be needed from Keystone . The
> previous
> >>>> attempt didn't get much traction and stalled out early in
> implementation. If
> >>>> we want to revisit this lets make sure we have the resources needed
> and
> >>>> spec(s) in progress / info on etherpads (similar to how the
> multitenancy
> >>>> stuff was handled at the last summit) as early as possible.
> >>>
> >>>
> >>> Why not centralize quota management via the python-openstackclient,
> what
> >>> is the benefit of getting keystone involved?
> >>
> >>
> >> Providing this through the openstack client in my opinion has the
> >> disadvantage that users which either use the REST API direct or write
> their
> >> own clients won't leverage it. I don't think it's a reasonable
> assumption
> >> that everybody will use python-openstackclient, is it?
> >>
> >> Said that, storing quotas in keystone poses a further challenge to the
> >> scalability of the system, which we shall perhaps address by using
> >> appropriate caching strategies and leveraging keystone notifications.
> Until
> >> we get that, I think that the openstack client will be the best way of
> >> getting a unified quota management experience.
> >>
> >> Salvatore
> >>
> >>
> >>>>
> >>>> Cheers,
> >>>> Morgan
> >>>>
> >>>> Sent via mobile
> >>>>
> >>>>
> >>>> On Friday, October 3, 2014, Salvatore Orlando <sorlando at nicira.com>
> >>>> wrote:
> >>>>>
> >>>>> Thanks Vish,
> >>>>>
> >>>>> this seems a very reasonable first step as well - and since most
> >>>>> projects would be enforcing quotas in the same way, the shared
> library would
> >>>>> be the logical next step.
> >>>>> After all this is quite the same thing we do with authZ.
> >>>>>
> >>>>> Duncan is expressing valid concerns which in my opinion can be
> addressed
> >>>>> with an appropriate design - and a decent implementation.
> >>>>>
> >>>>> Salvatore
> >>>>>
> >>>>> On 3 October 2014 18:25, Vishvananda Ishaya <vishvananda at gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> The proposal in the past was to keep quota enforcement local, but to
> >>>>>> put the resource limits into keystone. This seems like an obvious
> first
> >>>>>> step to me. Then a shared library for enforcing quotas with decent
> >>>>>> performance should be next. The quota calls in nova are extremely
> >>>>>> inefficient right now and it will only get worse when we try to add
> >>>>>> hierarchical projects and quotas.
> >>>>>>
> >>>>>> Vish
> >>>>>>
> >>>>>> On Oct 3, 2014, at 7:53 AM, Duncan Thomas <duncan.thomas at gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Taking quota out of the service / adding remote calls for quota
> >>>>>>> management is going to make things fragile - you've somehow got to
> >>>>>>> deal with the cases where your quota manager is slow, goes away,
> >>>>>>> hiccups, drops connections etc. You'll also need some way of
> >>>>>>> reconciling actual usage against quota usage periodically, to
> detect
> >>>>>>> problems.
> >>>>>>>
> >>>>>>> On 3 October 2014 15:03, Salvatore Orlando <sorlando at nicira.com>
> >>>>>>> wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Quota management is currently one of those things where every
> >>>>>>>> openstack
> >>>>>>>> project does its own thing. While quotas are obviously managed in
> a
> >>>>>>>> similar
> >>>>>>>> way for each project, there are subtle differences which
> ultimately
> >>>>>>>> result
> >>>>>>>> in lack of usability.
> >>>>>>>>
> >>>>>>>> I recall that in the past there have been several calls for
> unifying
> >>>>>>>> quota
> >>>>>>>> management. The blueprint [1] for instance, hints at the
> possibility
> >>>>>>>> of
> >>>>>>>> storing quotas in keystone.
> >>>>>>>> On the other hand, the blazar project [2, 3] seems to aim at
> solving
> >>>>>>>> this
> >>>>>>>> problem for good enabling resource reservation and therefore
> >>>>>>>> potentially
> >>>>>>>> freeing openstack projects from managing and enforcing quotas.
> >>>>>>>>
> >>>>>>>> While Blazar is definetely a good thing to have, I'm not entirely
> >>>>>>>> sure we
> >>>>>>>> want to make it a "required" component for every deployment.
> Perhaps
> >>>>>>>> single
> >>>>>>>> projects should still be able to enforce quota. On the other hand,
> >>>>>>>> at least
> >>>>>>>> on paper, the idea of making Keystone "THE" endpoint for managing
> >>>>>>>> quotas,
> >>>>>>>> and then letting the various project enforce them, sounds
> promising
> >>>>>>>> - is
> >>>>>>>> there any reason for which this blueprint is stalled to the point
> >>>>>>>> that it
> >>>>>>>> seems forgotten now?
> >>>>>>>>
> >>>>>>>> I'm coming to the mailing list with these random questions about
> >>>>>>>> quota
> >>>>>>>> management, for two reasons:
> >>>>>>>> 1) despite developing and using openstack on a daily basis I'm
> still
> >>>>>>>> confused by quotas
> >>>>>>>> 2) I've found a race condition in neutron quotas and the fix is
> not
> >>>>>>>> trivial.
> >>>>>>>> So, rather than start coding right away, it might probably make
> more
> >>>>>>>> sense
> >>>>>>>> to ask the community if there is already a known better approach
> to
> >>>>>>>> quota
> >>>>>>>> management - and obviously enforcement.
> >>>>>>>>
> >>>>>>>> Thanks in advance,
> >>>>>>>> Salvatore
> >>>>>>>>
> >>>>>>>> [1]
> https://blueprints.launchpad.net/keystone/+spec/service-metadata
> >>>>>>>> [2] https://wiki.openstack.org/wiki/Blazar
> >>>>>>>> [3]
> https://review.openstack.org/#/q/project:stackforge/blazar,n,z
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> OpenStack-dev mailing list
> >>>>>>>> OpenStack-dev at lists.openstack.org
> >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Duncan Thomas
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> OpenStack-dev mailing list
> >>>>>>> OpenStack-dev at lists.openstack.org
> >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> OpenStack-dev mailing list
> >>>>>> OpenStack-dev at lists.openstack.org
> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>>>
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> OpenStack-dev mailing list
> >>>> OpenStack-dev at lists.openstack.org
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev at lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> > --
> > Davanum Srinivas :: https://twitter.com/dims
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141014/4142437c/attachment.html>