Open Stack

On 08/05/2015 02:33 PM, Ryan Hallisey wrote:
> Tagging kolla so the kolla community also sees it.
> Pardon the top posting.
> 
> -Ryan
> 
> ----- Original Message -----
> From: "Dan Prince" <dprince at redhat.com>
> To: "openstack-dev" <openstack-dev at lists.openstack.org>
> Sent: Wednesday, August 5, 2015 2:29:13 PM
> Subject: [openstack-dev] [TripleO] [Puppet] Deploying OpenStack with Puppet modules on Docker with Heat
> 
> Hi,
> 
> There is a lot of interest in getting support for container based
> deployment within TripleO and many different ideas and opinions on how
> to go about doing that.
> 
> One idea on the table is to use Heat to help orchestrate the deployment
> of docker containers. This would work similar to our tripleo-heat
> -templates implementation except that when using docker you would swap
> in a nested stack template that would configure containers on
> baremetal. We've even got a nice example that shows what a
> containerized TripleO overcloud might look like here [1]. The approach
> outlines how you might use kolla docker containers alongside of the
> tripleo-heat-templates to do this sort of deployment.
> 
> This is all cool stuff but one area of concern is how we do the actual
> configuration of the containers. The above implementation relies on
> passing environment variables into kolla built docker containers which
> then self configure all the required config files and start the
> service. This sounds like a start... but creating (and maintaining)
> another from scratch OpenStack configuration tool isn't high on my list
> of things to spend time on. Sure there is already a kolla community
> helping to build and maintain this configuration tooling (mostly
> thinking config files here) but this sounds a bit like what tripleo
> -image-elements initially tried to do and it turns out there are much
> more capable configuration tools out there.
> 
> Since we are already using a good bit of Puppet in tripleo-heat
> -templates the idea came up that we would try to configure Docker
> containers using Puppet. Again, here there are several ideas in the
> Puppet community with regards to how docker might best be configured
> with Puppet. Keeping those in mind we've been throwing some ideas out
> on an etherpad here [2] that describes using Heat for orchestration,
> Puppet for configuration, and Kolla docker images for containers.
> 
> A quick outline of the approach is:
> 
> -Extend the heat-container-agent [3] that runs os-collect-config and
> all the required hooks we require for deployment. This includes docker
> -compute, bash scripts, and Puppet. NOTE: As described in the etherpad
> I've taken to using DIB to build this container. I found this to be
> faster from a TripleO development baseline.
> 
> -To create config files the heat-container-agent would run a puppet
> manifest for a given role and generate a directory tree of config files
> (/var/lib/etc-data for example).

I have a few questions:

* when do you run puppet? before starting the container so we can
generate a configuration file?
* so iiuc, Puppet is only here to generate OpenStack configuration files
and we noop all other operations. Right?
* from a Puppet perspective, I really prefer this approach:
https://review.openstack.org/#/c/197172/ where we assign tags to
resources so we can easily modify/drop Puppet resources using our
modules. What do you think (for long term)?
* how do you manage multiple configuration files? (if a controller is
running multiple nova-api containers with different configuration files?

Once I understand a bit more where we go, I'll be happy to help to make
it happen in our modules, we already have folks deploying our modules
with containers, I guess we can just talk and collaborate here.
Also, I'll be interested to bringing containers support in our CI, but
that's a next step :-)

Thanks Dan for this work,

> 
> -We then run a docker-compose software deployment that mounts those
> configuration file(s) into a read only volume and uses them to start
> the containerized service.
> 
> The approach could look something like this [4]. This nice thing about
> this is that it requires no modification to OpenStack Puppet modules.
> We can use those today, as-is. Additionally, although Puppet runs in
> the agent container we've created a mechanism to set all the resources
> to noop mode except for those that generate config files. And lastly,
> we can use exactly the same role manifest for docker that we do for
> baremetal. Lots of re-use here... and although we are disabling a lot
> of Puppet functionality in setting all the non-config resources to noop
> the Kolla containers already do some of that stuff for us (starting
> services, etc.).
> 
> ----
> 
> All that said (and trying to keep this short) we've still got a bit of
> work to do around wiring up externally created config files to kolla
> build docker containers. A couple of issues are:
> 
> -The external config file mechanism for Kolla containers only seems to
> support a single config file. Some services (Neutron) can have multiple
> files. Could we extend the external config support to use multiple
> files?
> 
> -If a service has multiple files kolla may need to adjust its service
> startup script to use multiple files. Perhaps a conf.d approach would
> work here?
> 
> -We are missing published version of some key kolla containers. Namely
> openvswitch and the neutron-openvswitch-agent for starters but I'd also
> like to have a Ceilometer agent and SNMP agent container as well so we
> have feature parity with the non-docker compute role.
> 
> Once we have solutions for the above I think we'll be very close to a
> fully dockerized compute role with TripleO heat templates. From there
> we can expand the idea to cover other roles within the tripleo-heat
> -templates too.
> 
> I'll stop there for now. Any ideas and thoughts appreciated.
> 
> Dan
> 
> -----
> 
> [1] https://review.openstack.org/#/c/178840/ (Containerized TripleO
> Overcloud.)
> [2] https://etherpad.openstack.org/p/tripleo-docker-puppet
> [3] http://git.openstack.org/cgit/openstack/heat
> -templates/log/hot/software-config/heat-container-agent
> [4] https://review.openstack.org/#/c/209505/  (Docker compute role
> configured via Puppet)
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-- 
Emilien Macchi

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150807/83baf2fd/attachment.pgp>