[Openstack] Jenkins vs SmokeStack tests & Gerrit merge blockers
Sean Dague
sdague at linux.vnet.ibm.com
Thu Jun 28 17:58:39 UTC 2012
On 06/28/2012 11:13 AM, Monty Taylor wrote:
<snip>
> Fundamentally though - we're at a point of trying to have our cake and
> eat it too. Either we want comprehensive testing of all of the unit
> tests, or we want to be careful about not making the test environment to
> hard for a developer to exactly mimic.
>
> I'm obviously on the side of having us have gating tests that some devs
> might not be able to do on their laptops - such as running the libvirt
> tests properly. We're working on cloud software - worst case scenario if
> there's an intractable problem, as dev can always spin up an ubuntu
> image somewhere.
I'm definitely in agreement here. Something as fundamental as libvirt is
to openstack really needs to see testing in the main test environment.
I really feel like this is almost the same conversation we had on IRC on
Friday about the mysql requirement. There is a deeper level of testing
that we can do if we start requiring more of the base OS in the jenkins
environment. That takes us out of a place where all the unit tests that
run in jenkins can be run in any python environment, but that's ok. More
validation here on the gate is worth that.
Maybe we need something that's different than the current @skip
semantics, some @skip_unless_gate (or similar), because a big piece of
this is also confusion about what the gate is actually testing, and what
it's skipping.
As the person involved in the patch that slipped through, I was as
surprised as anyone else that it landed, but had an issue in a real
libvirt test case.
-Sean
--
Sean Dague
IBM Linux Technology Center
email: sdague at linux.vnet.ibm.com
alt-email: sldague at us.ibm.com
More information about the Openstack
mailing list