Open Stack

So whats needed to build out that 'framework'?

Is it just dispatching provision requests to nova, and then seeing when 
the quota becomes out of sync, and then backtracking through logs and 
notifications (+- other) to then figure out the root cause?

Or is that framework some kind of local functional testing built-in to 
nova itself, something that can work without probing via the REST 
endpoints...?

IMHO it'd be nice to have something like rally scenarios that probe the 
quota (via REST or not) but with post-validation that can be done on the 
post scenario run and/or post request run. The post-validation ensures 
the quota is as expected, using some locally computed 'expected' quota 
that nova should also have locally computed, if those two computed 
'expected' quotas are different, then backtracking/log analysis... needs 
to occur to figure out when the computed values started to diverge (and 
to figure out why they diverged); rinse and repeat that and u probably 
have a pretty powerful validation framework...

Vilobh Meshram wrote:
> I am highly supportive for the idea of Nova Quota sub-team, for
> something as complex as Quota, as it helps to move quickly on reviews
> and changes.
>
> Agree with John, a test framework to test quotas will be helpful and can
> be one of the first task the Nova Quota sub team can focus on as that
> will lay the foundation for whether the bugs mentioned here
> http://bit.ly/1Pbr8YL are valid or not.
>
> Having worked in the area of Quotas for a while now by introducing
> features like Cinder Nested Quota Driver [1] [2] I strongly feel that
> something like a Nova Quota sub-team will definitely help. Mentioning
> about Cinder Quota driver since it was accepted in Mitaka design summit
> that Nova Nested Quota Driver[3] would like to pursue the route taken by
> Cinder.  Since Nested quota is a one part of Quota subsystem and working
> in small team helped to iterate quickly for Nested Quota
> patches[4][5][6][7] so IMHO forming a Nova quota subteam will help.
>
> Melanie,
>
> If you can share the details of the bug that Joe mentioned, to reproduce
> quota bugs locally, it would be helpful.
>
> -Vilobh (irc: vilobhmm)
>
> [1] Code : https://review.openstack.org/#/c/205369/
> [2] Blueprint :
> https://blueprints.launchpad.net/cinder/+spec/cinder-nested-quota-driver
> [3] Nova Nested Quota Spec : https://review.openstack.org/#/c/209969/
> [4] https://review.openstack.org/#/c/242568/
> [5] https://review.openstack.org/#/c/242500/
> [6] https://review.openstack.org/#/c/242514/
> [7] https://review.openstack.org/#/c/242626/
>
>
> On Mon, Nov 30, 2015 at 10:59 AM, melanie witt <melwittt at gmail.com
> <mailto:melwittt at gmail.com>> wrote:
>
>     On Nov 26, 2015, at 9:36, John Garbutt <john at johngarbutt.com
>     <mailto:john at johngarbutt.com>> wrote:
>
>     >  A suggestion in the past, that I like, is creating a nova functional
>     >  test that stress tests the quota code.
>     >
>     >  Hopefully that will be able to help reproduce the error.
>     >  That should help prove if any proposed fix actually works.
>
>     +1, I think it's wise to get some data on the current state of
>     quotas before choosing a redesign. IIRC, Joe Gordon described a test
>     scenario he used to use to reproduce quota bugs locally, in one of
>     the launchpad bugs. If we could automate something like that, we
>     could use it to demonstrate how quotas currently behave during
>     parallel requests and try things like disabling reservations. I also
>     like the idea of being able to verify the effects of proposed fixes.
>
>     -melanie (irc: melwitt)
>
>
>
>
>
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev