Open Stack

On Feb 24, 2015, at 9:47, Sean Dague <sean at dague.net> wrote:

> I'm happy if there are other theories about how we do these things,
> being the first functional test in the python-novaclient tree that
> creates and destroys real resources, there isn't an established pattern
> yet. But I think doing all CLI calls in CLI tests is actually really
> cumbersome, especially in the amount of output parsing code needed if
> you are going to setup any complicated resource structure.

I think I'm in agreement with the pattern you describe.

I imagine having a set of functional tests for the API, that don't do any CLI calls at all. With that we test that the API works properly. Then have a separate set for the CLI, which only calls CLI for the command being tested, everything else to set up and tear down the test done by API calls. This would be done with the rationale that because the entire API functionality is tested separately, we can safely use it for setup/teardown with the intent to isolate the CLI test to the command being tested and avoid introducing side effects from the CLI commands.

But I suppose one could make the same argument for using CLI everywhere (if they are all tested, they can also be trusted not to introduce side effects). I tend to favor using the API because it's the most bare bones setup/teardown we could use. At the same time I understand the idea of performing an entire test using the CLI, as a way of replicating the experience a real user might have using the CLI, from start to end. I don't think I feel strongly either way.

For the --poll stuff, I agree the API should have it and the CLI uses it. And with and without "poll" functionality should be tested separately, API and CLI.

melanie (melwitt)




-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150224/97773423/attachment.pgp>