Open Stack

Hi All,

My coworker and I would like to start filling out some gaps in api coverage that we see in the functional integration tests. We have one patch up for review (https://review.openstack.org/#/c/219025/). We got a comment saying that any new stack creation will prolong the testing cycle. We agree with that and it got us thinking about a few things -

  1.  We are planning on adding tests for the following api's: event api's, template api's, software config api's, cancel stack updates, check stack resources and show resource data. These are the api's that we saw aren't covered in our current integration tests. Please let us know if you feel we need tests for these upstream, if we're missing something or if it's already covered somewhere.
  2.  To conserve the creation of stacks would it make sense to add one test and then under that we could call sub methods that will run tests against that stack. So something like this:def _test_template_apis()

def _test_softwareconfig_apis()

def _test_event_apis()

def test_event_template_softwareconfig_apis(self):

stack_id = self.stack_create(...)

self._test_template_apis(stack_id)

self._test_event_apis(stack_id)

self._test_softwareconfig_apis(stack_id)

  3.  The current tests are divided into two folders - scenario and functional. To help with organization - under the functional folder, would it make sense to add an 'api' folder, 'resource' folder and 'misc folder? Here is what we're thinking about where each test can be put:
     *   API folder - test_create_update.py, test_preview.py

     *

Resource folder - test_autoscaling.py, test_aws_stack.py, test_conditional_exposure.py, test_create_update_neutron_port.py, test_encryption_vol_type.py, test_heat_autoscaling.py, test_instance_group.py, test_resource_group.py, test_software_config.py, test_swiftsignal_update.py

     *

Misc folder - test_default_parameters.py, test_encrypted_parameter.py, test_hooks.py, test_notifications.py, test_reload_on_sighup.py, test_remote_stack.py, test_stack_tags.py, test_template_resource.py, test_validation.py

  4.  Should we add to our README? For example, I see that we use TestResource as a resource in some of our tests but we don't have an explanation of how to set that up. I'd also like add explanations about the pre-testhook and post-testhook file and how that works and what each line does/what test it's attached to.
  5.  For the tests that we're working on, should we be be adding a blueprint or task somewhere to let everybody know that we're working on it so there is no overlap?
  6.  From our observations, we think it would be beneficial to add more comments to the existing tests.  For example, we could have a minimum of a short blurb for each method.  Comments?
  7.  Should we add a 'high level coverage' summary in our README?  It could help all of us know at a high level where we are at in terms of which resources we have tests for and which api's, etc.

Let us know what you all think!

Thank you!
Sabeen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150911/5657e83a/attachment.html>