Open Stack

On Thu, Jun 11, 2015 at 2:45 PM, Salvatore Orlando <sorlando at nicira.com> wrote:
> I have been then following a different approach. And a set of patches,
> including a devref one [2], if up for review [3]. This hardly completes the
> job: more work is required on the testing side, both as unit and functional
> tests.
>
> As for the spec, since I honestly would like to spare myself the hassle of
> rewriting it, I would kindly ask our glorious drivers team if they're ok
> with me submitting a spec in the shorter format approved for Liberty without
> going through the RFE process, as the spec is however in the Kilo backlog.

It took me a second read through to realize that you're talking to me
among the drivers team.  Personally, I'm okay with this and our
currently documented policy seems to allow for this until Liberty-1.

I just hope that this isn't an indication that we're requiring too
much in this new RFE process and scaring potential filers away.  I'm
trying to learn how to write good RFEs, so let me give it a shot:

  Summary:  "Need robust quota enforcement in Neutron."

  Further Information:  "Neutron can allow exceeding the quota in
certain cases.  Some investigation revealed that quotas in Neutron are
subject to a race where parallel requests can each check quota and
find there is just enough left to fulfill its individual request.
Each request proceeds to fulfillment with no more regard to the quota.
When all of the requests are eventually fulfilled, we find that they
have exceeded the quota."

Given my current knowledge of the RFE process, that is what I would
file as a bug in launchpad and tag it with 'rfe.'

> For testing I wonder what strategy do you advice for implementing functional
> tests. I could do some black-box testing and verifying quota limits are
> correctly enforced. However, I would also like to go a bit white-box and
> also verify that reservation entries are created and removed as appropriate
> when a reservation is committed or cancelled.
> Finally it would be awesome if I was able to run in the gate functional
> tests on multi-worker servers, and inject delays or faults to verify the
> systems behaves correctly when it comes to quota enforcement.

Full black box testing would be impossible to achieve without multiple
workers, right?  We've proposed adding multiple worker processes to
the gate a couple of times if I recall including a recent one to .
Fixing the failures has not yet been seen as a priority.

I agree that some whitebox testing should be added.  It may sound a
bit double-entry to some but I don't mind, especially given the
challenges around block box testing.  Maybe Assaf can chime in here
and set us straight.

> Do these kinds of test even make sense? And are they feasible at all? I
> doubt we have any framework for injecting anything in neutron code under
> test.

Dunno.

> Finally, please note I am using DB-level locks rather than non-locking
> algorithms for making reservations. I can move to a non-locking algorithm,
> Jay proposed one for nova for Kilo, and I can just implement that one, but
> first I would like to be convinced with a decent proof (or sort of) that the
> extra cost deriving from collision among workers is overshadowed by the cost
> for having to handle a write-set certification failure and retry the
> operation.

Do you have a reference describing the algorithm Jay proposed?

> Please advice.
>
> Regards,
> Salvatore
>
> [1]
> http://specs.openstack.org/openstack/neutron-specs/specs/kilo-backlog/better-quotas.html
> [2] https://review.openstack.org/#/c/190798/
> [3]
> https://review.openstack.org/#/q/project:openstack/neutron+branch:master+topic:bp/better-quotas,n,z
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>