Open Stack

Improving the QA test suite in multi-region would allow us to catch cases where a change has been made without considering the implications of multi-region support.

It would not however identify cases such as missing command line options. Given that the test suite identifies a set of operations with an expected result, if the scenario has been missed, it would not have a test case and therefore it would not be detected. Catching these is more of a code/spec review problem.

I'd suggest as a first step to do a quick analysis with the infra test team

- on what multi-region scenarios are tested in the gate currently
- what are the gaps with the current CLIs / APIs
- raise bugs where there are gaps
- what additional test cases need to be implemented to catch regressions

Tim

> -----Original Message-----
> From: Mathieu Gagné [mailto:mgagne at internap.com]
> Sent: 18 August 2015 23:48
> To: openstack-operators at lists.openstack.org
> Subject: [Openstack-operators] Improving quality of multi-region support
> 
> Hi,
> 
> This email is a follow-up on the discussion we are currently having in the
> LDT (Large Deployment Team) session at the OpenStack Ops Meetup. [1]
> 
> We collectively identified a lack test coverage around multi-region support.
> A region being the concept found in Keystone catalog, not the
> geographically concept.
> 
> Issues identified revolved around configs changes (without proper release
> notes), lack of support in OpenStack clients [2] or maybe OpenStack clients
> within services themselves.
> 
> Some issues got fixed after bug reports were opened, this is great.
> 
> However I feel like most were found out too late in the process when it
> could have been detected much earlier in the QA process.
> 
> I would like to start a discussion around multi-region test coverage and how
> we can improve the overall quality around multi-region support.
> 
> 
> [1] https://etherpad.openstack.org/p/PAO-ops-large-deployments
> [2] https://bugs.launchpad.net/python-ceilometerclient/+bug/1439553
> 
> --
> Mathieu
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators