Open Stack

On 03/02/2016 07:34 AM, Ivan Kolodyazhny wrote:
> Sean,
> 
> I've mentioned above, that current tempest job runs ~1429 tests and only
> about 10 of them uses cinderclient. It tooks a lot of time without any
> benefits for cinder, e.g.: tests like tempest.api.network.* verifies
> Neutron, not python-cinderclient.

We can say that about a lot of things in that stack. For better or
worse, that's where our testing is. It's a full stack same set of tests
against all these components which get used. The tempest.api.network
tests are quite quick. The biggest time hitters in the runs are scenario
tests, many of which are volumes driven.

2016-02-12 19:07:46.277 |
tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot[compute,id-7b6860c2-afa3-4846-9522-adeb38dfbe08,network]
 193.523
2016-02-12 19:07:46.277 |
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern[compute,id-557cd2c2-4eb8-4dce-98be-f86765ff311b,image,smoke,volume]
            150.766
2016-02-12 19:07:46.277 |
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern[compute,id-557cd2c2-4eb8-4dce-98be-f86765ff311b,image,smoke,volume]
          136.834
2016-02-12 19:07:46.278 |
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic[compute,id-e79f879e-debb-440c-a7e4-efeda05b6848,network]
           107.045
2016-02-12 19:07:46.278 |
tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac[compute,id-9178ad42-10e4-47e9-8987-e02b170cc5cd,network]
                            101.252
2016-02-12 19:07:46.278 |
tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless[compute,id-cf1c4425-766b-45b8-be35-e2959728eb00,network]
                  99.041
2016-02-12 19:07:46.278 |
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
                        96.954
2016-02-12 19:07:46.278 |
tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance[compute,id-c1b6318c-b9da-490b-9c67-9339b627271f,image,network,volume]
        95.120
2016-02-12 19:07:46.278 |
tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario[compute,id-bdbb5441-9204-419d-a225-b4fdbfb1a1a8,image,network,volume]
           86.165
2016-02-12 19:07:46.278 |
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern[compute,id-608e604b-1d63-4a82-8e3e-91bc665c90b4,image,network]
                          85.422


If you would like to pitch in on an optimization strategy for all the
components, that would be great. But this needs to be thought about in
those terms. It would be great to stop testing 2 versions of cinder API
in every run, for instance. That would be super helpful to everyone as
those Boot from volume tests take over 2 minutes each.

	-Sean

-- 
Sean Dague
http://dague.net