Open Stack

On 10/11/2013 02:03 PM, Alessandro Pilotti wrote:
>
>
>
>
> On Oct 11, 2013, at 19:29 , Russell Bryant <rbryant at redhat.com 
> <mailto:rbryant at redhat.com>>
>  wrote:
>
>> On 10/11/2013 12:04 PM, John Griffith wrote:
>>>
>>> [... snip ...]
>
> Talking about new community involvements, newcomers are getting very 
> frustrated to have to wait for weeks to get a meaningful review and I 
> cannot blame them if they don't want to get involved anymore after the 
> first patch!
> This makes appear public bureocracy here in eastern Europe a 
> lightweight process in comparison! :-)
>
> Let me add another practical reason about why a separate OpenStack 
> project would be a good idea:
>
> Anytime that we commit a driver specific patch, a lot of Tempests 
> tests are executed on Libvirt and XenServer (for Icehouse those will 
> be joined by another pack of CIs, including Hyper-V).
> On the jenkins side, we have to wait for regression tests that have 
> nothing to do with the code that we are pushing. During the H3 push, 
> this meant waiting for hours and hoping not to have to issue the 100th 
> "recheck / revery bug xxx".
>
> A separate project would obviously include only the required tests and 
> be definitely more lightweight, offloading quite some work from the 
> SmokeStack / Jenkins job for everybody's happiness.
>
>
I'm glad you brought this up. There are two issues here, both discussed 
by the qe/infra groups and others at the Havana summit and after.

How do you/we know which regression tests have nothing to do with the 
code changed in a particular patch? Or that the answer won't change 
tomorrow? The only way to do that is to assert dependencies and 
non-dependencies between components that will be used to decide which 
tests should be run for each patch. There was a lively discussion (with 
me taking your side initially) at the summit and it was decided that a 
generic "wasting resources" argument was not sufficient to introduce 
that fragility and so we would run the whole test suite as a gate on all 
projects. That decision was to be revisited if resources became a problem.

As for the 100th recheck, that is a result of the recent introduction of 
parallel tempest runs before the Havana rush. It was decided that the 
benefit in throughput from drastically reduced gate job times outweighed 
the pain of potentially doing a lot of rechecks. For the most part the 
bugs being surfaced were real OpenStack bugs that were showing up due to 
the new "stress" of parallel test execution. This was a good thing, 
though certainly painful to all. With hindsight I'm not sure if that was 
the right decision or not.

This is just an explanation of what has happened and why. There are 
obviously costs and benefits of being tightly bound to the project.

  -David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131011/e363148e/attachment.html>