Open Stack

I've been reading a bunch of the existing Dockerfiles, and I have two humble 
requests:


1. It would be good if the "interesting" code came from python sdist/bdists 
rather than rpms.

This will make it possible to rebuild the containers using code from a private 
branch or even unsubmitted code, without having to go through a redhat/rpm 
release process first.

I care much less about where the python dependencies come from. Pulling them 
from rpms rather than pip/pypi seems like a very good idea, given the relative 
difficulty of caching pypi content and we also pull in the required C, etc 
libraries for free.


With this in place, I think I could drop my own containers and switch to 
reusing kolla's for building virtual testing environments.  This would make me 
happy.


2. I think we should separate out "run the server" from "do once-off setup".

Currently the containers run a start.sh that typically sets up the database, 
runs the servers, creates keystone users and sets up the keystone catalog.  In 
something like k8s, the container will almost certainly be run multiple times 
in parallel and restarted numerous times, so all those other steps go against 
the service-oriented k8s ideal and are at-best wasted.

I suggest making the container contain the deployed code and offer a few thin 
scripts/commands for entrypoints.  The main replicationController/pod _just_ 
starts the server, and then we have separate pods (or perhaps even non-k8s 
container invocations) that do initial database setup/migrate, and post-
install keystone setup.

I'm open to whether we want to make these as lightweight/independent as 
possible (every daemon in an individual container), or limit it to one per 
project (eg: run nova-api, nova-conductor, nova-scheduler, etc all in one 
container).   I think the differences are run-time scalability and resource-
attribution vs upfront coding effort and are not hugely significant either way.

Post-install catalog setup we can combine into one cross-service setup like 
tripleO does[1].  Although k8s doesn't have explicit support for batch tasks 
currently, I'm doing the pre-install setup in restartPolicy: onFailure pods 
currently and it seems to work quite well[2].

(I'm saying "post install catalog setup", but really keystone catalog can 
happen at any point pre/post aiui.)

[1] https://github.com/openstack/tripleo-incubator/blob/master/scripts/setup-endpoints
[2] https://github.com/anguslees/kube-openstack/blob/master/kubernetes-in/nova-db-sync-pod.yaml

-- 
 - Gus