[openstack-dev] [openstack-qa] Graduation Requirements + Scope of Tempest
malini.kamalambal at RACKSPACE.COM
Mon Mar 24 16:31:35 UTC 2014
On 3/21/14 3:49 PM, "Rochelle.RochelleGrober" <rochelle.grober at huawei.com>
>> From: Malini Kamalambal [mailto:malini.kamalambal at RACKSPACE.COM]
>> We are talking about different levels of testing,
>> 1. Unit tests - which everybody agrees should be in the individual
>> 2. System Tests - 'System' referring to (& limited to), all the
>> that make up the project. These are also the functional tests for the
>> 3. Integration Tests - This is to verify that the OS components
>> well and don't break other components -Keystone being the most obvious
>> example. This is where I see getting the maximum mileage out of
>> I see value in projects taking ownership of the System Tests - because
>> the project is not 'functionally ready', it is not ready to integrate
>> other components of Openstack.
>> But for this approach to be successful, projects should have diversity
>> the team composition - we need more testers who focus on creating these
>> This will keep the teams honest in their quality standards.
>+1000 I love your approach to this. You are right. Functional tests
>for the project, that exist in an environment, but that exercise the
>intricacies of just the project aren't there for most projects, but
>really should be. And these tests should be exercised against new code
>before the code enters the gerrit/Jenkins stream. But, as Malini points
>out, it's at most a dream for most projects as the test developers just
>aren't part of most projects.
>> As long as individual projects cannot guarantee functional test
>> we will need more tests in Tempest.
>> But that will shift focus away from Integration Testing, which can be
>> ONLY in Tempest.
>+1 This is also an important point. If functional testing belonged to
>the projects, then most of these tests would be run before a tempest test
>was ever run and would not need to be part of the integrated tests,
>except as a subset that demonstrate the functioning integration with
>> Regardless of whatever we end up deciding, it will be good to have
>> discussions sooner than later.
>> This will help at least the new projects to move in the right
>Maybe a summit topic? How do we push functional testing into the project
That is a good idea
I just suggested a session for the ATL summit. See
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
More information about the OpenStack-dev