Open Stack

On Tue, Jul 16, 2013 at 10:58:11AM +0200, Koderer, Marc wrote:
> On 07/15/2013 at 19:30, Matthew Treinish [mailto:mtreinish at kortar.org]
> > 
> > On Mon, Jul 15, 2013 at 06:32:05PM +0200, Giulio Fidente wrote:
> > > On 07/15/2013 04:21 PM, Matthew Treinish wrote:
> > > >On Mon, Jul 15, 2013 at 01:23:02PM +0200, Koderer, Marc wrote:
> > > >>Could you do me a favor and update all work items that are already
> > done? Like " Add an entry in tox.ini to run run_stress.py" - I found
> > something here in the mailing list indicating that it's already
> > done....
> > > >
> > > >Yes I took care of adding the base tox job with:
> > > >https://review.openstack.org/#/c/35394/
> > >
> > > I see we have to pass every stress test name as argument; are there
> > > plans to run this in some automated job?
> > 
> > So at the same time I added the tox job I added a periodic jenkins job
> > to run the tox job:
> > 
> > https://jenkins.openstack.org/job/periodic-tempest-devstack-vm-stress/
> > 
> > and the logs are here:
> > 
> > http://logs.openstack.org/periodic/periodic-tempest-devstack-vm-stress/
> > 
> > However, when I looked at things this morning I noticed that it's not
> > actually running anything, I've got a fix pending:
> > 
> > https://review.openstack.org/#/c/37099/
> > 
> > >
> > > Shall we think about some "filtering" for those or share some
> > > agreement over which stress tests we want to run automatically?
> > 
> > Yeah the tox job I put there was just a starting point. I figured as
> > the tests got more complex (and plentiful) we could just update the
> > job. So whatever gets run for that tox job is what runs in the periodic
> > job. Hopefully run_tests.py will get a bit more sane about how we run
> > things too.
> 
> Let's enhance the stress tests with a new actions and we could define a
> good mixture of them for automatic testing. 
> 
> What do you mean with "more sane"? Could you propose a better interface? 

I just think the whole definition with a JSON file is a little strange. I
would also like to see an easier way to run the different actions easily
serially or in parallel (if it makes sense). If you look at the stress tox job
I had to call run_stress multiple times, once for each JSON action that existed.
I think it would be better if you could pass a list of actions you want to run
and have run_stress handle run them serially. (or something like that). 

It also might make sense to add an optional flag to run_tests.sh so the stress
suite is runnable from both tox and run_tests.

> 
> > 
> > >
> > > Worth a blueprint?
> > 
> > I think this falls under the scope of the existing stress_test
> > blueprint
> 
> Yes, it's in the current scope.
> 

-Matt Treinish