Open Stack

On 06/08/15 06:29, Dan Prince wrote:
> Hi,
>
> There is a lot of interest in getting support for container based
> deployment within TripleO and many different ideas and opinions on how
> to go about doing that.
>
> One idea on the table is to use Heat to help orchestrate the deployment
> of docker containers. This would work similar to our tripleo-heat
> -templates implementation except that when using docker you would swap
> in a nested stack template that would configure containers on
> baremetal. We've even got a nice example that shows what a
> containerized TripleO overcloud might look like here [1]. The approach
> outlines how you might use kolla docker containers alongside of the
> tripleo-heat-templates to do this sort of deployment.
>
> This is all cool stuff but one area of concern is how we do the actual
> configuration of the containers. The above implementation relies on
> passing environment variables into kolla built docker containers which
> then self configure all the required config files and start the
> service. This sounds like a start... but creating (and maintaining)
> another from scratch OpenStack configuration tool isn't high on my list
> of things to spend time on. Sure there is already a kolla community
> helping to build and maintain this configuration tooling (mostly
> thinking config files here) but this sounds a bit like what tripleo
> -image-elements initially tried to do and it turns out there are much
> more capable configuration tools out there.
>
> Since we are already using a good bit of Puppet in tripleo-heat
> -templates the idea came up that we would try to configure Docker
> containers using Puppet. Again, here there are several ideas in the
> Puppet community with regards to how docker might best be configured
> with Puppet. Keeping those in mind we've been throwing some ideas out
> on an etherpad here [2] that describes using Heat for orchestration,
> Puppet for configuration, and Kolla docker images for containers.
>
> A quick outline of the approach is:
>
> -Extend the heat-container-agent [3] that runs os-collect-config and
> all the required hooks we require for deployment. This includes docker
> -compute, bash scripts, and Puppet. NOTE: As described in the etherpad
> I've taken to using DIB to build this container. I found this to be
> faster from a TripleO development baseline.
>
> -To create config files the heat-container-agent would run a puppet
> manifest for a given role and generate a directory tree of config files
> (/var/lib/etc-data for example).
>
> -We then run a docker-compose software deployment that mounts those
> configuration file(s) into a read only volume and uses them to start
> the containerized service.
>
> The approach could look something like this [4]. This nice thing about
> this is that it requires no modification to OpenStack Puppet modules.
> We can use those today, as-is. Additionally, although Puppet runs in
> the agent container we've created a mechanism to set all the resources
> to noop mode except for those that generate config files. And lastly,
> we can use exactly the same role manifest for docker that we do for
> baremetal. Lots of re-use here... and although we are disabling a lot
> of Puppet functionality in setting all the non-config resources to noop
> the Kolla containers already do some of that stuff for us (starting
> services, etc.).
This sounds like a viable approach, my only suggestion would be for 
there to be an option to build a puppet-container-agent which contains 
only puppet (not the heat hook too). This could allow the 
openstack-puppet and kolla communities to collaborate quickly without 
pulling in the whole tripleo stack. Then some simple docker-compose (or 
whatever) templates could be written to bring up puppet-container-agent 
with a given manifest & hieradata, then bring up a single node kolla 
container based cloud. This would be useful for CI and local development 
of the puppet modules supporting containers.

Then heat-container-agent can be puppet-container-agent plus the heat 
hook tooling.

> ----
>
> All that said (and trying to keep this short) we've still got a bit of
> work to do around wiring up externally created config files to kolla
> build docker containers. A couple of issues are:
>
> -The external config file mechanism for Kolla containers only seems to
> support a single config file. Some services (Neutron) can have multiple
> files. Could we extend the external config support to use multiple
> files?
>
> -If a service has multiple files kolla may need to adjust its service
> startup script to use multiple files. Perhaps a conf.d approach would
> work here?
>
> -We are missing published version of some key kolla containers. Namely
> openvswitch and the neutron-openvswitch-agent for starters but I'd also
> like to have a Ceilometer agent and SNMP agent container as well so we
> have feature parity with the non-docker compute role.
>
> Once we have solutions for the above I think we'll be very close to a
> fully dockerized compute role with TripleO heat templates. From there
> we can expand the idea to cover other roles within the tripleo-heat
> -templates too.
>
> I'll stop there for now. Any ideas and thoughts appreciated.
>
> Dan
>
> -----
>
> [1] https://review.openstack.org/#/c/178840/ (Containerized TripleO
> Overcloud.)
> [2] https://etherpad.openstack.org/p/tripleo-docker-puppet
> [3] http://git.openstack.org/cgit/openstack/heat
> -templates/log/hot/software-config/heat-container-agent
> [4] https://review.openstack.org/#/c/209505/  (Docker compute role
> configured via Puppet)
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev