Open Stack

On Tue, Nov 25, 2014 at 10:03:41AM -0500, Sean Dague wrote:
> As we are trying to do smart disaggregation of tests in the gate, I
> think it's important to figure out which test configurations seem to be
> actually helping, and which aren't. As the swift team has long had a
> functional test job, this seems like a good place to start. (Also the
> field deploy / upgrade story on Swift is probably one of the best of any
> OpenStack project, so removing friction is probably in order.)
> 
> gate-swift-pep8	SUCCESS in 1m 16s
> gate-swift-docs	SUCCESS in 1m 48s
> gate-swift-python27	SUCCESS in 3m 24s
> check-tempest-dsvm-full	SUCCESS in 56m 51s
> check-tempest-dsvm-postgres-full	SUCCESS in 54m 53s
> check-tempest-dsvm-neutron-full	SUCCESS in 1h 06m 09s
> check-tempest-dsvm-neutron-heat-slow	SUCCESS in 31m 18s
> check-grenade-dsvm	SUCCESS in 39m 33s
> gate-tempest-dsvm-large-ops	SUCCESS in 29m 34s
> gate-tempest-dsvm-neutron-large-ops	SUCCESS in 22m 11s
> gate-swift-tox-func	SUCCESS in 2m 50s (non-voting)
> check-swift-dsvm-functional	SUCCESS in 17m 12s
> check-devstack-dsvm-cells	SUCCESS in 15m 18s
> 
> 
> I think in looking at that it's obvious that:
> * check-devstack-dsvm-cells
> * check-tempest-dsvm-postgres-full
> * gate-tempest-dsvm-large-ops
> * gate-tempest-dsvm-neutron-large-ops
> * check-tempest-dsvm-neutron-full
> 
> Provide nothing new to swift, the access patterns on the glance => swift
> interaction aren't impacted on any of those, neither is the heat / swift
> resource tests or volumes / swift backup tests.

So I agree with all of this and think removing the jobs is fine, except the
postgres job and the neutron jobs do test the glance->swift access pattern, and
do run the heat swift tests. But, it does raise the bigger question which was
brought up in Darmstadt and again at summit on having a single gating
configuration. Maybe we should just switch to doing that now and finally drop
the postgres job completely.

> 
> check-tempest-dsvm-neutron-heat-slow	doesn't touch swift either (it's
> actually remarkably sparse of any content).

I think we probably should be removing this job from everywhere, we've slowly
been whittling away the job because it doesn't seem to be capable of being run
reliably. This also doesn't run any swift resource tests, in it's current form
it runs 6 neutron resource tests and thats it.

> 
> Which kind of leaves us with 1 full stack run, and the grenade job. Have
> those caught real bugs? Does there remain value in them? Have other
> teams that rely on swift found those to block regressions?

So I think we'll need these at a minimum for the time being. Giving our current
project structure (and governance requirements) having jobs that test things
work together I feel is important. I know we've caught issues with glance->swift
layer with these jobs in the past as well as other bugs as well as bugs in swift
before. (although they're very infrequent compared to other projects)

> 
> Let's figure out what's helpful, and what's not, and purge out all the
> non helpful stuff.
> 

-Matt Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141125/4617eebb/attachment.pgp>