Open Stack

(Trying to summarize discussions from earlier on IRC)

On Mon, May 25, 2015 at 06:54:43PM +0000, Steven Dake (stdake) wrote:
>Hey fellow Kolla devs,
>
>With Sam’s recent change to add build from source as an option and build from Debuntian binaries as an option, we will end up in a situation where our gate will take 4+ hours to build all of the images and run the functional tests.  I would like to separate each major distro and source type with a separate functional gate, for example:
>
>centos-rdo
>fedora-rdo
>ubuntu-binary
>debian-bianry
>centos-source
>fedora-source
>debian-source
>ubuntu-source
>
>I propose separating each of these as a separate non-voting check job.  What needs to happen in our image building scripts, our functional tests, and the project-config repo to make this happen?

Sam said he was working on a patch the allowed CLI args to be passed in 
to set the prefix. Then it appears that tox supports passing arguments 
to underlying tests, so a new argument could be passed to 
test_images.py, and that file be modified to pass the different prefix 
to build-all-docker-images. (The project-config repo would be nearly 
identical, just with new jobs executing tox with the different 
argument.)

I really wish we were somehow using caching, though there is one 
downside. If caching were used and a network resource goes down that was 
cached, the build would succeed, which could be confusing. Then again 
that very con could be considered a pro as being more robust to third 
party failures.

Today on IRC it was mentioned that pushing images would take hours, so I 
think we should leverage Docker's trusted builds and not even bother 
trying to push images from our test run (another potential solution).  
The flow would look something like:
submitted to gerrit
approved
gating performed
commit permitted to land in repo
github commit hook triggers trusted build
trusted build updated to latest and available for all to download from 
docker registry

Looking into this further, we'd need infrastructure help to get 
permissions on github to create the webhooks.

Once we get to this point we should be able to do image pulls of trusted 
builds, greatly accelerating the build process. However, if this is 
deemed too risky, the trusted builds would at least allow community 
users to stay up to date easily without any long build times.

>I’d like to make our current functional gate voting asap, but don’t want to block build from source to make that happen.

ASAP? I thought we discussed leaving the job to run for a while before 
making it voting. But if voting is to be turned on in the near term, 
obviously we'd start without caching and just using the centos images 
for now. As far as I can tell, the review is ready: 
https://review.openstack.org/#/c/183417/

Thoughts?