Open Stack

On Fri, Jul 14, 2017 at 2:17 AM, Flavio Percoco <flavio at redhat.com> wrote:
>
> Greetings,
>
> As some of you know, I've been working on the second phase of TripleO's
> containerization effort. This phase if about migrating the docker based
> deployment onto Kubernetes.
>
> These phase requires work on several areas: Kubernetes deployment, OpenStack
> deployment on Kubernetes, configuration management, etc. While I've been
> diving
> into all of these areas, this email is about the second point, OpenStack
> deployment on Kubernetes.
>
> There are several tools we could use for this task. kolla-kubernetes,
> openstack-helm, ansible roles, among others. I've looked into these tools
> and
> I've come to the conclusion that TripleO would be better of by having
> ansible
> roles that would allow for deploying OpenStack services on Kubernetes.
>
> The existing solutions in the OpenStack community require using Helm. While
> I
> like Helm and both, kolla-kubernetes and openstack-helm OpenStack projects,
> I
> believe using any of them would add an extra layer of complexity to TripleO,
> which is something the team has been fighting for years years - especially
> now
> that the snowball is being chopped off.
>
> Adopting any of the existing projects in the OpenStack communty would
> require
> TripleO to also write the logic to manage those projects. For example, in
> the
> case of openstack-helm, the TripleO team would have to write either ansible
> roles or heat templates to manage - install, remove, upgrade - the charts
> (I'm
> happy to discuss this point further but I'm keepping it at a high-level on
> purpose for the sake of not writing a 10k-words-long email).
>
> James Slagle sent an email[0], a couple of days ago, to form TripleO plans
> around ansible. One take-away from this thread is that TripleO is adopting
> ansible more and more, which is great and it fits perfectly with the
> conclusion
> I reached.
>
> Now, what this work means is that we would have to write an ansible role for
> each service that will deploy the service on a Kubernetes cluster. Ideally
> these
> roles will also generate the configuration files (removing the need of
> puppet
> entirely) and they would manage the lifecycle. The roles would be isolated
> and
> this will reduce the need of TripleO Heat templates. Doing this would give
> TripleO full control on the deployment process too.
>
> In addition, we could also write Ansible Playbook Bundles to contain these
> roles
> and run them using the existing docker-cmd implementation that is coming out
> in
> Pike (you can find a PoC/example of this in this repo[1]).
>
> Now, I do realize the amount of work this implies and that this is my
> opinion/conclusion. I'm sending this email out to kick-off the discussion
> and
> gather thoughts and opinions from the rest of the community.
>
> Finally, what I really like about writing pure ansible roles is that ansible
> is
> a known, powerfull, tool that has been adopted by many operators already.
> It'll
> provide the flexibility needed and, if structured correctly, it'll allow for
> operators (and other teams) to just use the parts they need/want without
> depending on the full-stack. I like the idea of being able to separate
> concerns
> in the deployment workflow and the idea of making it simple for users of
> TripleO
> to do the same at runtime. Unfortunately, going down this road means that my
> hope of creating a field where we could collaborate even more with other
> deployment tools will be a bit limited but I'm confident the result would
> also
> be useful for others and that we all will benefit from it... My hopes might
> be a
> bit naive *shrugs*

Of course I'm biased since I've been (a little) involved in that work
but I like the idea of :

- Moving forward with our containerization. docker-cmd will help us
for sure for this transition (I insist on the fact TripleO is a
product that you can upgrade and we try to make it smooth for our
operators), so we can't just trash everything and switch to a new
tool. I think the approach that we're taking is great and made of baby
steps where we try to solve different problems.
- Using more Ansible - the right way - when it makes sense : with the
TripleO containerization, we only use Puppet for Configuration
Management, managing a few resources but not for orchestration (or not
all the features that Puppet provide) and for Data Binding (Hiera). To
me, it doesn't make sense for us to keep investing much in Puppet
modules if we go k8s & Ansible. That said, see the next point.
- Having a transition path between TripleO with Puppet and TripleO
with apbs and have some sort of binding between previous hieradata
generated by TripleO & a similar data binding within Ansible playbooks
would help. I saw your PoC Flavio, I found it great and I think we
should make https://github.com/tripleo-apb/ansible-role-k8s-keystone/blob/331f405bd3f7ad346d99e964538b5b27447a0ebf/provision-keystone-apb/tasks/hiera.yaml
optional when running apbs, and allow to provide another format (more
Ansiblish) to let folks not using TripleO to use it. We also should
target this new format and switch service by service in TripleO to use
this new format, as long as apbs support both. I think that way we can
step by step migrate to use Ansible for configuration management.

There are some things to figure out:
- We kind of found out solutions for OpenStack services - great - now
what do we do for services like MySQL, Apache, etc. Do we have
"standard" and "community-supported" apbs? Do we need to create some?
- Where the apbs should live? IMO in OpenStack and IMO not in big tent
for now, under no umbrella.
- Since we use Puppet modules which don't only make configuration
management but also some orchestration (like creating keystone
endpoints, etc) - where should we put this logic? +1 for apbs using
clean Ansible code (and not bash templating).
- How we can help our vendors to whom we asked them to write Puppet
modules to deploy their software (Contrail, Nuage, etc). We still
might need some sort or "running puppet from ansible" for some
software we wouldn't have apbs as quickly as we would need.

I hope I didn't divert the discussion but here's my feedback and food
for thoughts.
Thanks for your work,
-- 
Emilien Macchi