Open Stack

On 11/22/2016 07:56 PM, John Dickinson wrote:
> 
> 
> On 22 Nov 2016, at 10:47, Andreas Jaeger wrote:
> 
>> When we (infra) changed the unit test jobs to not set up databases by
>> default, we created special python-db and tox-db jobs that set up both
>> MySQL and PostgreSQL databases. And that complicated the setup of those
>> projects and lead to problems like setting projects up via bindep for
>> both databases even if one was used.
>>
>> We had last week an IRC discussion [1] and came up with the following
>> approach:
>>
>>   Projects can use a tools/test-setup.sh script that is called from
>>   our unit test (tox, python27, python34, python35) targets. The
>>   script is executed as root and should set up the needed databases -
>>   or whatever is needed. The script needs to reside in the repository -
>>   and thus might need to get backported to older branches.
>>
>>   This setup should be used for any kind of repo specific unit test
>>   setup.
>>
>>   Projects are suggested to add to their developer documents, e.g. the
>>   README or CONTRIBUTING or TESTING file, the usage of
>>   tools/testsetup.sh. Developers should be able to use the script to
>>   set up prerequisites for unit tests locally.
>>
>>   Long term goal is for projects to not use the -db jobs anymore, new
>>   changes for them should not be accepted.
>>
>> This is implemented in project-config [2], an example usage in
>> nodepool [3,4], which leads to a cleanup [5].
>>
>> Further investigation shows that the special searchlight setup can be
>> solved with the same approach (searchlight change [6], project-config
>> [7]). Here it's interesting to note that moving the setup in the
>> repository, found a problem: The repo needs elasticsearch 1 for
>> liberty and 2 for newer branches, this can now be done inside the
>> repository.
>>
>> The xfs TMPDIR setup of swift [2] could been done in general this way as
>> well but that change needs to set TMPDIR for the unittests, passing
>> information from the set up builder to the tox builder. This is
>> currently not possible using only the proposed solution, and so would
>> still require a custom tox job. Alternative, this could be changed with
>> some other way of passing the value of TMPDIR between these different
>> invocations.
> 
> (actually link [10])

Indeed, thanks for correcting.

> This sounds like a great idea that I would have loved to have in place a few weeks ago!
> 
> One question: is there a limit to which tox environments will call this in-repo script? Above you list a few, but Swift and other projects have repo-specific environments that would need the setup as well. How will that work?

So far it's the templates I mentioned. The custom swift jobs are not
setup this way and I'm fine with removing the custom jobs and use the
default jobs again.

In general, if additional jobs need the change from [2], we can add them
- let's discuss this one by one in a review,

Andreas

> 
>>
>> Today, a change was proposed [8,9] that would setup docker for kolla
>> and kolla-ansible. I suggest to not merge it and instead use the same
>> approach here.
>>
>> Credits for the proposal go to Jeremy - and this got triggered by
>> comments by Jim. Thanks!
>>
>> Andreas
>>
>> [1]
>> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-11-17.log.html#t2016-11-17T15:07:38
>>
>> [2] https://review.openstack.org/399105
>> [3] https://review.openstack.org/399079
>> [4] https://review.openstack.org/399177
>> [5] https://review.openstack.org/399180
>> [6] https://review.openstack.org/399159
>> [7] https://review.openstack.org/399169
>> [8] https://review.openstack.org/400128
>> [9] https://review.openstack.org/400474
>> [10] https://review.openstack.org/394600
>> -- 
>>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>>    GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>>        HRB 21284 (AG Nürnberg)
>>     GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
       HRB 21284 (AG Nürnberg)
    GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126