Open Stack

On Sat, Jul 11, 2015 at 11:35:16AM +1200, Robert Collins wrote:
> On 9 July 2015 at 10:52, Jeremy Stanley <fungi at yuggoth.org> wrote:
> > On 2015-07-09 10:37:17 +1200 (+1200), Robert Collins wrote:
> >> So - I'm looking to:
> >>
> >> A) have a discussion and identify any issues with moving testr out of
> >> the venvs. (Note: this doesn't mean stop using it, just removing it
> >> from test-requirements.txt, in the same way that tox isn't in
> >> test-requirements.txt).
> >> B) Capture that in a spec if its non-trivial.
> >> C) find volunteers to make it happen.
> >
> > D) keep reminding developers to install it on their systems when
> > they ask why they can't run tests
> 
> We can make the os-testr which *is* installed in the venv look for
> testr in the PATH and error with a good explanation when its not
> there.

I guess we could do this, it would be very simple to implement and I'm not
necessarily opposed to it, if that's the direction everyone wants to go. Right
now ostestr only interacts with testr through subprocess, which makes the
implementation of something like this quite simple.

But, I have 2 concerns right now: First was that my medium/long term my plan was
to use testr's ui layer to implement ostestr's cli instead of the dirty
subprocess hack I did to get things done faster. Doing this would require either
breaking out of the venv to get the testr imports (which I wouldn't really want
to do), require all the tox venv use system site-packages (which I don't think
is really a good idea, and wouldn't solve the python version issue) or making
sure testr was installed in the venv too.

The other is ostestr isn't universally used right now, only a few projects are
using the ostestr CLI. os-testr is only really being installed in the venv
everywhere because projects use subunit-trace, which is also part of os-testr,
via a pretty_tox.sh script. So to make this a solution we would have to migrate
everything over to ostestr first. Which wouldn't really be an issue and it's
something I want to do eventually.

TBH, I tend to agree with everyone else's opinion elsewhere on this thread.
While it might be viewed as an anti-pattern for how testr was originally
intended to be used it doesn't change that this is pretty much the model used
and expected by everything and everyone using testr (or any test runner) in
OpenStack. I would view doing something like this as more of a workaround than
just fixing the storage engine to be compatible with multiple python versions.

The whole argument for making testr live outside of the venv and being an
implicit dependency like tox is based around tracking the results between the
tox venvs right? If we decoupled the database and other result storage from
testr a bit and made that the external platform independent piece wouldn't we
maintain everything this thread is trying to address without having the issues
we're seeing now or requiring that everyone change their usage model and
expectations?

-Matt Treinish

> 
> > E) keep reminding developers to upgrade to a newer version when they
> > start running into bugs which aren't exhibited in our CI
> 
> We already have to do that (because we don't hard-require -U) - we
> could easily enough cross-check the version of testr from os-testr too
> and issue appropriate errors.
> 
> > I think the original decision to install it inside the tox
> > virtualenv was because:
> >
> > 1. it made migrating from nose easier because we didn't have to add
> > new steps in the basic workflow
> >
> > 2. it's one less thing developers need to know to install on their
> > systems
> >
> > 3. it makes sure a new enough version is being used (matching the
> > version used in our CI)
> >
> > I'm not arguing against the change, but think it's worth
> > acknowledging the (perhaps marginal) ongoing costs of the solution
> > and asking whether they're outweighed by the ongoing cost of working
> > around the problem.
> 
> Certainly there are tradeoffs.
> 
> -Rob
> 
> -- 
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Converged Cloud
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150710/fd91e23c/attachment.pgp>