Open Stack

2015-06-17 12:38 GMT+09:00 Yuiko Takada <yuikotakada0313 at gmail.com>:
>
> Then, as you and Matt and Dimitry talked about this on IRC few days ago,
> We can add Ironic/Ironic-inspector tests into Tempest still, right?
> So that I've started to implement a test in Tempest,
> but I'm facing another issue.
> As you know, Ironic API has microversions, and Ironic-inspector can run
> with microversion > 1.6.
> But currently there is no feature testing specific Ironic API microversions
> on Tempest, right?
>
> So that, we have to think about some solutions.
>
> (1) Make testing specific Ironic API microversions on Tempest possible
> adam_g is posting this patch set.
> https://review.openstack.org/166386
>
> (2)Using tempest_lib instead of adding tests into Tempest
> Is tempest_lib available already?
> Or do we need to wait for something will be merged?

I guess the above question seems multiple factors are mixed.
You want to test ironic-inspector behaviors by
  * using ironic-inspector REST APIs directly without Ironic
  * using Ironic REST APIs which need newer microversion
right?

For the first test, you can implement without considering microversion.
The test just calls ironic-inspector REST APIs directly and checks its behavior.
You can implement the test on Tempest/ironic-inspector repository.
Current tempest-lib seems enough to implement tests in
ironic-inspector repository as features, but it is better to wait for
Tempest's external interface spec[1] approval.
It is trying to define directory structure of Tempest-like tests on
each project repository and Tempest will discover tests based on the
directory structure and run them.
So if implementing tests on ironic-inspector repository before the
spec approval, you will need to change the directory structure again
in the future.

For the second test, microversions support is necessary on Tempest
side and adam_g's patch seems good for implementing it.
My main concern of microversions tests is how to run multiple
microversions on the gate.
We have discussed that in Nova design session of Vancouver summit and
the conclusion is
 * Minimum microversion
 * Maximum microversion
 * Interesting microversions
as the gate test.
IMO "Interesting microversions" would be the last microversions of
each release(Kilo, Liberty, ..) I feel.
I have qa-spec[2] for testing microversions on the gate, but that is
not complete yet.
That will affect how to specify/run microversions tests on Tempest.
So now I'm not sure yet the way to specify microversion on current
adam_g's patch is the best.

So my recommendation/hope is that we concentrate on Tempest's external
interface spec[1] and make it better together, then we can implement
Tempest-like tests on each repository after that.
As the next step, we will test microversions on the same way between
projects based on conclusion of the spec[2].

> (3)Make Ironic-inspector available even if microversion < 1.6
> Dmitry is posting this patch set.
> https://review.openstack.org/192196
> # I don't mean asking you to review this, don't worry :p

I've reviewed it already :)

Thanks
Ken Ohmichi

---
[1]: https://review.openstack.org/#/c/184992/
[2]: https://review.openstack.org/#/c/169126/