On Mon, Mar 21, 2016, at 09:32 AM, Armando M. wrote: > Do you have an a better insight of job runtimes vs jobs in other > projects? > Most of the time in the job runtime is actually spent setting the > infrastructure up, and I am not sure we can do anything about it, unless > we > take this with Infra. I haven't done a comparison yet buts lets break down the runtime of a recent successful neutron full run against neutron master [0]. Basic host setup takes 65 seconds. Start of job to 2016-03-17 22:14:27.397 [1] Workspace setup takes 520 seconds. 2016-03-17 22:14:27.397 [1] to 2016-03-17 22:23:07.429 [2] Devstack takes 1205 seconds. 2016-03-17 22:23:18.760 [3] to 2016-03-17 22:43:23.339 [4] Loading old tempest subunit streams takes 155 seconds. 2016-03-17 22:43:23.340 [5] to 2016-03-17 22:45:58.061 [6] Tempest takes 1982 seconds. 2016-03-17 22:45:58.201 [7] to 2016-03-17 23:19:00.117 [8] Then we spend the rest of the test time (76 seconds) cleaning up. 2016-03-17 23:19:00.117 [8] to end of job. Note that I haven't accounted for all of the time used and instead of focused on the major steps that use the most time. Also it is Monday morning and some of my math may be off. [0] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/ [1] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_14_27_397 [2] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_23_07_429 [3] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_23_18_760 [4] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_43_23_339 [5] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_43_23_340 [6] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_45_58_061 [7] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_22_45_58_201 [8] http://logs.openstack.org/18/294018/3/check/gate-tempest-dsvm-neutron-full/1cce1e8/console.html.gz#_2016-03-17_23_19_00_117 One big takeaway from this is that the vast majority of the time is spent in devstack and tempest not in the infrastructure setup. You should be able to dig into both the devstack setup and tempest test runtimes and hopefully speed things up. Hopefully this gives you enough information to get started into digging on this. Clark