[openstack-dev] oslo.db 1.5.0 requirements.txt requires testresources/testscenarios

Doug Hellmann doug at doughellmann.com
Fri Mar 6 20:48:49 UTC 2015

On Fri, Mar 6, 2015, at 03:28 PM, Matt Riedemann wrote:
> I filed this as a bug:
> https://bugs.launchpad.net/oslo.db/+bug/1429233
> To hopefully summarize, the new provision stuff in oslo.db requires 
> testresources and the provision stuff is a public interface for other 
> projects to hook in DB testing (a fixture).
> This moved testscenarios out of test-requirements into requirements.txt 
> and also added testresources to requirements.txt.
> The main problem I have with this is it's essentially test code in a 
> non-test path and as a packager that builds and has to get legal 

It sounds like we may have 2 issues. If we have runtime code importing
something that is supposed to be a test fixture, we should fix that by
refactoring the code in oslo.db. The question of how to manage those
dependencies is separate from that.

> clearance on runtime dependencies based on requirements.txt, I'd 
> normally have to package and ship testresources/testscenarios now to use 
> oslo.db >= 1.5.0, otherwise stevedore will be unhappy at startup when 
> there are missing requirements.

stevedore should be using the pkg_resources option to ignore
dependencies, so it should no longer complain. kg_resources used
elsewhere might still complain. Do you have a traceback, or are you
anticipating issues?

> I could workaround this in my packaging by removing the test* entries 
> from requirements.txt (which is probably what I'll have to do in the 
> short term), but this still seems like an issue that needs to be fixed 
> because now all projects that require oslo.db at runtime will also have 
> testresources/testscenarios in their runtime import path, and that 
> grosses me out.

The test fixtures are a public part of the API of the library. We
therefore want to include the dependencies so that when the library is
installed it is usable. We could alternately decide to treat the test
code like we do drivers, and say that there is a separate set of
dependencies to be installed when the test code is needed. It's not
quite the same case, but it's probably close enough.

> I don't want to bang the revert drum on the change that introduced this 
> since it took several months to get merged.  I'm thinking more long-term 
> though, that this kind of thing should live in a separate proejct, e.g. 
> oslo.db.fixtures, something like that, so oslo.db can require that in 
> test-requirements.txt and so can any other project, e.g. nova, that's 
> using those fixtures for it's DB testing.  That's obviously not 
> something that's going to happen in Kilo though at this point since 
> we're coming up on dependency freeze.

That would effectively double the number of libraries in Oslo, though,
because we would have a separate test library for every project. It also
means the test fixtures become much less useful, because they are meant
to be able to be updated as the internals of the library change, but if
they are in a separate repository it will be more complex to do that.

> I'm bringing this to the mailing list to get more attention to the 
> topic.  There is also a spec in oslo for defining external library 
> requirements [1] with some discussion happening there.
> [1] https://review.openstack.org/#/c/157135/
> -- 
> Thanks,
> Matt Riedemann
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list