Open Stack

On 01/07/2016 03:01 PM, Jim Rollenhagen wrote:
> On Thu, Jan 07, 2016 at 02:41:12PM -0500, Sean Dague wrote:
>> On 01/07/2016 02:09 PM, Jim Rollenhagen wrote:
>>> Hi all,
>>>
>>> A change to global-requirements[1] introduces mimic, which is an http
>>> server that can mock various APIs, including nova and ironic, including
>>> control of error codes and timeouts. The ironic team plans to use this
>>> for testing python-ironicclient without standing up a full ironic
>>> environment.
>>>
>>> Here's the catch - mimic is built on twisted. I know twisted was
>>> previously removed from OpenStack (or at least people said "pls no", I
>>> don't know the full history). We didn't intend to stealth-introduce
>>> twisted back into g-r, but it was pointed out to me that it may appear
>>> this way, so here I am letting everyone know. lifeless pointed out that
>>> when tests are failing, people may end up digging into mimic or twisted
>>> code, which most people in this community aren't familiar with AFAIK,
>>> which is a valid point though I hope it isn't required often.
>>>
>>> So, the primary question here is: do folks have a problem with adding
>>> twisted here? We're holding off on Ironic changes that depend on this
>>> until this discussion has happened, but aren't reverting the g-r change
>>> until we decide one way or another.
>>>
>>> // jim
>>>
>>> [1] https://review.openstack.org/#/c/220268/
>>
>> What is the advantage of running another server like this over using
>> requests-mock (which is used by other OpenStack projects for testing
>> today)? The only difference here seems to be that you actually execute
>> requests code in one case and not in the other.
>>
>> Requests-mock debugging when things go wrong seems a bit simpler.
>>
>> This is less about twisted and more about trying to not introduce yet
>> another way to mock code in the tree that people need to understand.
>>
>> 	-Sean
>
> We'd be using this for functional tests, not unit, where we can't really
> inject mocks. The idea is that we could run a full functional suite
> against either mimic or a full ironic environment, just by changing a
> test setting.

I don't really see the point of a separate project like Mimic that has a 
whole bunch of reimplementations (mocked out) of all sorts of OpenStack 
(and RAX-specific) API services. It's just a great way to introduce a 
larger surface area for bugs to creep in -- since you have to keep the 
Mimic interfaces up to date with the real interfaces. Better to keep 
something like this -- if it is TRULY needed -- in-tree with the API 
service itself, so that the chances of divergence are reduced. This is 
similar to the fakevirt driver in Nova. It's in tree for good reason: 
when someone changes the virt driver interface, the fakevirt driver goes 
boom and needs to be changed in a corresponding fashion in the same patch.

What value does a functional test against an HTTP API service that does 
nothing (other than introduce greater surface area for bugs) actually 
offer over unit tests anyway?

-jay