Open Stack

On 22 November 2016 18:56, John Dickinson wrote:

>
> On 22 Nov 2016, at 10:47, Andreas Jaeger wrote:
> 
> > When we (infra) changed the unit test jobs to not set up databases by
> > default, we created special python-db and tox-db jobs that set up both
> > MySQL and PostgreSQL databases. And that complicated the setup of
> > those projects and lead to problems like setting projects up via
> > bindep for both databases even if one was used.
> >
> > We had last week an IRC discussion [1] and came up with the following
> > approach:
> >
> >   Projects can use a tools/test-setup.sh script that is called from
> >   our unit test (tox, python27, python34, python35) targets. The
> >   script is executed as root and should set up the needed databases -
> >   or whatever is needed. The script needs to reside in the repository -
> >   and thus might need to get backported to older branches.
> >
> >   This setup should be used for any kind of repo specific unit test
> >   setup.
> >
> >   Projects are suggested to add to their developer documents, e.g. the
> >   README or CONTRIBUTING or TESTING file, the usage of
> >   tools/testsetup.sh. Developers should be able to use the script to
> >   set up prerequisites for unit tests locally.
> >
> >   Long term goal is for projects to not use the -db jobs anymore, new
> >   changes for them should not be accepted.
> >
> > This is implemented in project-config [2], an example usage in
> > nodepool [3,4], which leads to a cleanup [5].
> >
> > Further investigation shows that the special searchlight setup can be
> > solved with the same approach (searchlight change [6], project-config
> > [7]). Here it's interesting to note that moving the setup in the
> > repository, found a problem: The repo needs elasticsearch 1 for
> > liberty and 2 for newer branches, this can now be done inside the
> > repository.
> >
> > The xfs TMPDIR setup of swift [2] could been done in general this way
> > as well but that change needs to set TMPDIR for the unittests, passing
> > information from the set up builder to the tox builder. This is
> > currently not possible using only the proposed solution, and so would
> > still require a custom tox job. Alternative, this could be changed
> > with some other way of passing the value of TMPDIR between these
> > different invocations.
> 
> (actually link [10])
> 
> This sounds like a great idea that I would have loved to have in place a few
> weeks ago!
> 
> One question: is there a limit to which tox environments will call this in-repo
> script? Above you list a few, but Swift and other projects have repo-specific
> environments that would need the setup as well. How will that work?
> 

+1 this is a really useful feature.

Further to John's question, would it be possible for the test-setup.sh script to be passed the job template name as an arg? That would allow customized setup for specific test jobs. Right now I don’t have a specific use case in Swift but I'm curious as to whether it is a possibility.

> 
> >
> > Today, a change was proposed [8,9] that would setup docker for kolla
> > and kolla-ansible. I suggest to not merge it and instead use the same
> > approach here.
> >
> > Credits for the proposal go to Jeremy - and this got triggered by
> > comments by Jim. Thanks!
> >
> > Andreas
> >
> > [1]
> > http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack
> > -infra.2016-11-17.log.html#t2016-11-17T15:07:38
> >
> > [2] https://review.openstack.org/399105
> > [3] https://review.openstack.org/399079
> > [4] https://review.openstack.org/399177
> > [5] https://review.openstack.org/399180
> > [6] https://review.openstack.org/399159
> > [7] https://review.openstack.org/399169
> > [8] https://review.openstack.org/400128
> > [9] https://review.openstack.org/400474
> > [10] https://review.openstack.org/394600
> > --
> >  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
> >   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> >    GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> >        HRB 21284 (AG Nürnberg)
> >     GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272
> > A126
> >
> >
> ________________________________________________________________
> ______
> > ____ OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev