<div dir="ltr">Hi, <div><br></div><div>It's still not clear for me, why we can't just add Rally jobs with scenarios related to specific project. </div><div>It will work quite fast and it will cover CLI (instantly)  with good integration/functional testing. </div><div><br></div><div><br></div><div>Best regards,</div><div>Boris Pavlovic </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 2, 2016 at 4:52 AM, Sean Dague <span dir="ltr"><<a href="mailto:sean@dague.net" target="_blank">sean@dague.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 03/02/2016 07:34 AM, Ivan Kolodyazhny wrote:<br>
> Sean,<br>
><br>
</span><span class="">> I've mentioned above, that current tempest job runs ~1429 tests and only<br>
> about 10 of them uses cinderclient. It tooks a lot of time without any<br>
> benefits for cinder, e.g.: tests like tempest.api.network.* verifies<br>
> Neutron, not python-cinderclient.<br>
<br>
</span>We can say that about a lot of things in that stack. For better or<br>
worse, that's where our testing is. It's a full stack same set of tests<br>
against all these components which get used. The tempest.api.network<br>
tests are quite quick. The biggest time hitters in the runs are scenario<br>
tests, many of which are volumes driven.<br>
<br>
2016-02-12 19:07:46.277 |<br>
tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot[compute,id-7b6860c2-afa3-4846-9522-adeb38dfbe08,network]<br>
 193.523<br>
2016-02-12 19:07:46.277 |<br>
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern[compute,id-557cd2c2-4eb8-4dce-98be-f86765ff311b,image,smoke,volume]<br>
            150.766<br>
2016-02-12 19:07:46.277 |<br>
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern[compute,id-557cd2c2-4eb8-4dce-98be-f86765ff311b,image,smoke,volume]<br>
          136.834<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic[compute,id-e79f879e-debb-440c-a7e4-efeda05b6848,network]<br>
           107.045<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac[compute,id-9178ad42-10e4-47e9-8987-e02b170cc5cd,network]<br>
                            101.252<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless[compute,id-cf1c4425-766b-45b8-be35-e2959728eb00,network]<br>
                  99.041<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]<br>
                        96.954<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance[compute,id-c1b6318c-b9da-490b-9c67-9339b627271f,image,network,volume]<br>
        95.120<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario[compute,id-bdbb5441-9204-419d-a225-b4fdbfb1a1a8,image,network,volume]<br>
           86.165<br>
2016-02-12 19:07:46.278 |<br>
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern[compute,id-608e604b-1d63-4a82-8e3e-91bc665c90b4,image,network]<br>
                          85.422<br>
<br>
<br>
If you would like to pitch in on an optimization strategy for all the<br>
components, that would be great. But this needs to be thought about in<br>
those terms. It would be great to stop testing 2 versions of cinder API<br>
in every run, for instance. That would be super helpful to everyone as<br>
those Boot from volume tests take over 2 minutes each.<br>
<div class="HOEnZb"><div class="h5"><br>
        -Sean<br>
<br>
--<br>
Sean Dague<br>
<a href="http://dague.net" rel="noreferrer" target="_blank">http://dague.net</a><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>