Open Stack

On 01/07/2016 03:32 PM, Jay Pipes wrote:
> On 01/07/2016 03:01 PM, Jim Rollenhagen wrote:
>> On Thu, Jan 07, 2016 at 02:41:12PM -0500, Sean Dague wrote:
>>> On 01/07/2016 02:09 PM, Jim Rollenhagen wrote:
>>>> Hi all,
>>>>
>>>> A change to global-requirements[1] introduces mimic, which is an http
>>>> server that can mock various APIs, including nova and ironic,
>>>> including
>>>> control of error codes and timeouts. The ironic team plans to use this
>>>> for testing python-ironicclient without standing up a full ironic
>>>> environment.
>>>>
>>>> Here's the catch - mimic is built on twisted. I know twisted was
>>>> previously removed from OpenStack (or at least people said "pls no", I
>>>> don't know the full history). We didn't intend to stealth-introduce
>>>> twisted back into g-r, but it was pointed out to me that it may appear
>>>> this way, so here I am letting everyone know. lifeless pointed out
>>>> that
>>>> when tests are failing, people may end up digging into mimic or
>>>> twisted
>>>> code, which most people in this community aren't familiar with AFAIK,
>>>> which is a valid point though I hope it isn't required often.
>>>>
>>>> So, the primary question here is: do folks have a problem with adding
>>>> twisted here? We're holding off on Ironic changes that depend on this
>>>> until this discussion has happened, but aren't reverting the g-r
>>>> change
>>>> until we decide one way or another.
>>>>
>>>> // jim
>>>>
>>>> [1] https://review.openstack.org/#/c/220268/
>>>
>>> What is the advantage of running another server like this over using
>>> requests-mock (which is used by other OpenStack projects for testing
>>> today)? The only difference here seems to be that you actually execute
>>> requests code in one case and not in the other.
>>>
>>> Requests-mock debugging when things go wrong seems a bit simpler.
>>>
>>> This is less about twisted and more about trying to not introduce yet
>>> another way to mock code in the tree that people need to understand.
>>>
>>>     -Sean
>>
>> We'd be using this for functional tests, not unit, where we can't really
>> inject mocks. The idea is that we could run a full functional suite
>> against either mimic or a full ironic environment, just by changing a
>> test setting.
>
> I don't really see the point of a separate project like Mimic that has
> a whole bunch of reimplementations (mocked out) of all sorts of
> OpenStack (and RAX-specific) API services. It's just a great way to
> introduce a larger surface area for bugs to creep in -- since you have
> to keep the Mimic interfaces up to date with the real interfaces.
> Better to keep something like this -- if it is TRULY needed -- in-tree
> with the API service itself, so that the chances of divergence are
> reduced. This is similar to the fakevirt driver in Nova. It's in tree
> for good reason: when someone changes the virt driver interface, the
> fakevirt driver goes boom and needs to be changed in a corresponding
> fashion in the same patch.
A tool like my OpenStackInABox could certainly benefit from the models
or services being provided by each project - aside from the complexity
that that adds to the installation of installing all the dependencies
related instead of just implementing a simple model with a file and
sqlite backend that has minimal dependencies...but I digress as I
haven't looked at how nova's fakevirt driver installs so may be that's
not as big an issue. I'll certainly look at it for use in
OpenStackInABox, but even there I'm aiming for a more complete scenario
where you can interact with Keystone, Nova, Swift, etc...(e.g auth
against a fake Keystone, use the token with the fake nova which
validates it against the same fake Keystone instance, same with a fake
Swift...).

> What value does a functional test against an HTTP API service that
> does nothing (other than introduce greater surface area for bugs)
> actually offer over unit tests anyway?

So to use Nova's fakevirt your project is limited to the same language
as nova - python, correct?

The main advantage of mimic is that it is language agnostic and multiple
moving parts (f.e taskflow) can interact with it.

$0.02

Ben