Open Stack

On 7/21/2016 2:03 AM, Bhor, Dinesh wrote:
> Hi Nova Devs,
>
>
>
> Many times, there are a number of data sets that we have to run the same
> tests on.
>
> And, to create a different test for each data set values is
> time-consuming and inefficient.
>
>
>
> Data Driven Testing [1] overcomes this issue. Data-driven testing (DDT)
> is taking a test,
>
> parameterizing it and then running that test with varying data. This
> allows you to run the
>
> same test case with many varying inputs, therefore increasing coverage
> from a single test,
>
> reduces code duplication and can ease up error tracing as well.
>
>
>
> DDT is a third party library needs to be installed separately and invoke the
>
> module when writing the tests. At present DDT is used in cinder and rally.

There are several projects using it:

http://codesearch.openstack.org/?q=ddt%3E%3D1.0.1&i=nope&files=&repos=

I first came across it when working a little in manila.

>
>
>
> To start with, I have reported this as a bug [2] and added initial patch
> [3] for the same,
>
> but couple of reviewers has suggested to discuss about this on ML as
> this is not a real bug.
>
> IMO this is not a feature implementation and it’s just a effort for
> simplifying our tests,
>
> so a blueprint will be sufficient to track its progress.
>
>
>
> So please let me know whether I can file a new blueprint or nova-specs
> to proceed with this.
>
>
>
> [1] http://ddt.readthedocs.io/en/latest/index.html
>
> [2] https://bugs.launchpad.net/nova/+bug/1604798
>
> [3] https://review.openstack.org/#/c/344820/
>
>
>
> Thank you,
>
> Dinesh Bhor
>
>
> ______________________________________________________________________
> Disclaimer: This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidential, and proprietary data. If you are not the intended recipient,
> please advise the sender by replying promptly to this email and then delete
> and destroy this email and any attachments without any further use, copying
> or forwarding.
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

I agree that it's not a bug. I also agree that it helps in some specific 
types of tests which are doing some kind of input validation (like the 
patch you've proposed) or are simply iterating over some list of values 
(status values on a server instance for example).

Using DDT in Nova has come up before and one of the concerns was hiding 
details in how the tests are run with a library, and if there would be a 
learning curve. Depending on the usage, I personally don't have a 
problem with it. When I used it in manila it took a little getting used 
to but I was basically just looking at existing tests and figuring out 
what they were doing when adding new ones.

I definitely think DDT is easier to use/understand than something like 
testscenarios, which we're already using in Nova.

-- 

Thanks,

Matt Riedemann