Open Stack

On 01/11/2016 03:49 PM, Serge Kovaleff wrote:
> Hi All,
>
> Last week I had a noble goal to write "one-more" functional test in Ironic.
> I did find a folder "func" but it was empty.
>
> Friends helped me to find a WIP patch
> https://review.openstack.org/#/c/235612/
>
> and here comes the question of this email: what approach we would like
> to implement:
> Option 1 - write infrastructure code that starts/configure/stops the
> services
> Option 2 - rely on installed DevStack and run the tests over it
>
> Both options have their Cons and Pros. Both options are implemented
> across the OpenStack umbrella.
> Option 1 - Glance, Nova, the patch above
> Option 2 - HEAT and my favorite at the moment.
>
> Any ideas?

I think we should eventually end up with supporting both standalone 
functional tests (#1) and tempest-based (#3).

A bit of context on #1. We've been using it in ironic-inspector since 
nearly its inception, when devstack plugins didn't exist and our project 
was on stackforge, so we had no way of implementing our dsvm gate. The 
basic idea is to start a full-featured service with mocked access to 
other services and simplified environment. We've written a decorator [1] 
that starts the service in __enter__ and stops in __exit__. It uses a 
temporary configuration file [2] with authentication disabled and 
database in temporary SQLite file. The service is started in a new green 
thread and exits when the test exits. We mock ironicclient with the 
usual 'mock' library to avoid requirements on a running ironic instance.

We do 2 kinds of tests: just an API test like [3] or full-flow 
introspection tests like [4]. In the latter case we first start 
introspection via API, verify that status is "in progress", then call 
the ramdisk callback endpoint with a fake data, and verify that 
introspection ends successfully.

Applying the same thing to ironic might be somewhat trickier. We can run 
conductor and API in the same process and use oslo.messaging fake driver 
[5] to avoid AMQP dependency. We'll have to use a fake network 
implementation and either mock glance or make sure we use local file 
system URL's for all images (IIRC we do support it).

Going further, if we create a simple fake IPA, we can even do a 
full-flow test with fake_agent driver. We will start deployment, make 
sure fake IPA was called with a right image, make it report success and 
see deployment finish. We might want to modify fake interfaces to record 
all calls, so that we can verify that boot interface was called properly.

[1] 
https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/test/functional.py#L358-L393
[2] 
https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/test/functional.py#L36-L51
[3] 
https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/test/functional.py#L249-L291
[4] 
https://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/test/functional.py#L177-L196
[5] http://docs.openstack.org/developer/oslo.messaging/drivers.html#fake

>
> Cheers,
> Serge Kovaleff
> http://www.mirantis.com <http://www.mirantis.com/>
> cell: +38 (063) 83-155-70
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>