Open Stack

On Thu, Jan 14, 2016 at 11:00:16AM +1300, Robert Collins wrote:
> On 8 January 2016 at 08:09, Jim Rollenhagen <jim at jimrollenhagen.com> wrote:
> > Hi all,
> >
> > A change to global-requirements[1] introduces mimic, which is an http
> > server that can mock various APIs, including nova and ironic, including
> > control of error codes and timeouts. The ironic team plans to use this
> > for testing python-ironicclient without standing up a full ironic
> > environment.
> 
> Whee that was a bit of a thread. Sorry for kicking this off on IRC and
> not chiming in until now.
> 
> As I read it the following points were made:
>   - this is a new an different testing thing
>   - which has the same interface skew issues as regular mocks
>   - and devs may need to hack twisted to fix issues with it in future
>   - but it is being used to target local devs, not gate jobs [today]
> 
> Interface skew can be handled a couple of ways. I'm a big fan of a
> technique (that I *think* came out of Google) of having a test fake
> layer which is substitutable by real things, so you can always run it
> in 'slow mode' to make sure things still work with real components.
> It's my understanding that with the way mimic is used, this can be
> done: run the same tests with real e.g. devstack brought up
> components. I think that that is good insurance, particularly if this
> is being done periodically rather than
> when-someone-remembers-or-has-an-issue.

Indeed, this is mimic's intent, and the way it's been used elsewhere
(e.g. by autoscale QA folks).

> Another way to mitigate interface skew is to have the publisher of an
> interface also publish a fast test fake for folk to use. I am also a
> fan of doing that, but as we haven't done so yet, I think its ok to go
> with what we have. If we couldn't run the same things with a real
> implementation, I would be more worried - because mocks of moving
> interfaces are fragile (e.g. mocking out oslo.config is fragile vs
> mocking out os.unlink).

So this made me think a bit and poke Glyph - turns out out-of-tree
plugins are being worked on by mimic; might be worth investigating
putting the mimic plugin for ironic in ironic's tree itself.

> Personally, I'm fine hacking on Twisted code - I quite like it
> (particularly good Twisted code :)) - and since folk seem ok with that
> aspect, great - thank you for raising the discussion here.

Totally; thanks for chiming in :)

// jim