Open Stack

On Fri, Jan 08, 2016 at 02:08:04PM -0500, Jay Pipes wrote:
> On 01/07/2016 07:38 PM, Jim Rollenhagen wrote:
snippity snip snip

> >We haven't made it a dep for anything yet, only added to g-r.
> 
> According to Dims, not to g-r, but to u-c, right Dims? Not sure if that
> makes functionally any difference, though (pun intended).

Both. https://review.openstack.org/#/c/220268/

This thread was originally about twisted, which is added to u-c with the
introduction of mimic.

> >However, now that you mention that, a really ambitious goal would be to
> >add a rabbit interface to mimic, and functionally test the API server
> >(that it sends the right messages, etc). Another would be to mimic
> >(Neutron, Glance) and test Ironic by itself.
> 
> So you would reimplement AMQP communication protocols using an in-memory
> data store for queues. Sounds like an even greater surface area for bugs to
> be introduced.
> 
> >Last, I frankly don't understand why there's
> >such heavy opposition to the ironic team using an additional tool for
> >testing.
> 
> Since you asked, I'll be blunt. This isn't a personal attack on you, Jim,
> though.
> 
> a) Because it fractures the testing and QA processes used by upstream
> contributors that work on OpenStack projects by requiring them to learn
> another system -- and one that potentially would require them to understand
> a whole new surface area for potential bugs

I don't think there's a large risk of needing to dig deep into mimic,
and especially twisted. If this does prove to be a problem, I'm happy to
remove it. However, we can't even explore what it would be like, or how
hard we would depend on mimic, without mimic being in g-r.

> b) Because it represents yet another RAX-driven divergence in the QA space.
> CloudCafe took essentially all of the RAX folks that were at one point
> working on Tempest and upstream QA and siloed them into a totally different
> organization, in true RAX fashion. Instead of pulling the OpenStack QA
> community along together, RAX QA continues to just do its own thing and
> there's still bitterness on the tips of tongues.

So, this isn't trying to replace anything. This is adding a different
way to run functional tests, that is *much* faster than standing up a
full ironic environment. This is helpful for developers that want to
quickly run tests before posting them to gerrit, people that need to
test in constrained environments, etc.

I'm 100% against doing things like Rackspace did with tempest and
cloudcafe, and I wouldn't be supporting this effort if I felt it was
similar. Here's how this went:

* Lekha started working on OnMetal QA, with a goal of doing some amount
  of upstream work as well.

* She's previously worked on projects (like autoscale) that interact
  with OpenStack APIs, and seeing the need to test without a full nova
  environment, built mimic.

* In talking with some of the other Ironic folks working on QA (from
  Intel, IBM, more), she presented mimic as something that may be useful
  for testing the client (and more). They (and I) agreed it was a neat
  idea worth trying.

* Jay offered to help with the global-requirements patch as it's
  something he's done before, and did the review grinding here.

* It finally landed, and lifeless asked me to bring up the Twisted
  conversation on the list. Note that this is not the "is mimic
  useful" conversation we're having now. Nobody remotely voiced
  concerns about using a new test tool until this thread happened.

Please do let me know if any of that seems nefarious; I hope it doesn't.

> 
> <snip>
> 
> >There was *more* than sufficient time for "I don't think this is useful"
> >to be posted on the review.
> 
> Sorry, this was the first I'd heard of it all.
> 
> > If anyone has a hard technical reason why
> >mimic should not be used to test an OpenStack project, I'm happy to
> >listen. Otherwise I'd like to work on actually getting things done.
> 
> I've listed my hard technical reason: it introduces, IMHO, too large of a
> surface area for bugs to creep in to the testing process versus the benefit
> it provides.

Again, I don't think that's any larger of a surface area than mocks in client
unit tests being incorrect, and we really won't know whether it causes
problems in reality.

// jim