Open Stack

On Thu, Jul 20, 2017 at 9:04 PM, Paul Belanger <pabelanger at redhat.com> wrote:
> On Thu, Jul 20, 2017 at 06:21:22PM -0400, James Slagle wrote:
>> Following up on the previous thread:
>> http://lists.openstack.org/pipermail/openstack-dev/2017-July/119405.html
>>
>> I wanted to share some work I did around the prototype I mentioned
>> there. I spent a couple days exploring this idea. I came up with a
>> Python script that when run against an in progress Heat stack, will
>> pull all the server and deployment metadata out of Heat and generate
>> ansible playbooks/tasks from the deployments.
>>
>> Here's the code:
>> https://github.com/slagle/pump
>>
>> And an example of what gets generated:
>> https://gist.github.com/slagle/433ea1bdca7e026ce8ab2c46f4d716a8
>>
>> If you're interested in any more detail, let me know.
>>
>> It signals the stack to completion with a dummy "ok" signal so that
>> the stack will complete. You can then use ansible-playbook to apply
>> the actual deloyments (in the expected order, respecting the steps
>> across all roles, and in parallel across all the roles).
>>
>> Effectively, this treats Heat as nothing but a yaml cruncher. When
>> using it with deployed-server, Heat doesn't actually change anything
>> on an overcloud node, you're only using it to generate ansible.
>>
>> Honestly, I think I will prefer the longer term approach of using
>> stack outputs. Although, I am not sure of the end goal of that work
>> and if it is the same as this prototype.
>>
> Sorry if this hasn't been asked before but why don't you removed all of your
> ansible-playbook logic out of heat and write them directly as native playbooks /
> roles? Then instead of having a tool that reads heat to then generate the
> playbooks / roles, you update heat just to directly call the playbooks? Any
> dynamic information about be stored in the inventory or using the --extra-vars
> on the CLI?

We must maintain backwards compatibility with our existing Heat based
interfaces (cli, api, templates). While that could probably be done
with the approach you mention, it feels like it would be much more
difficult to do so in that you'd need to effectively add back on the
compatibility layer once the new pristine native ansible
playbooks/roles were written. And it seems like it would be quite a
lot of heat template work to translate existing interfaces to call
into the new playbooks.

Even then, any new playbooks written from scratch would have to be
flexible enough to accommodate the old interfaces. On the surface, it
feels like you may end up sacrificing a lot of your goals in your
playbooks so you can maintain backwards compatibility anyways.

The existing interface must be the first class citizen. We can't break
those contracts, so we need ways to quickly iterate towards ansible.
Writing all new native playbooks sounds like just writing a new
OpenStack installer to me, and then making Heat call that so that it's
backwards compatible.

The focus on the interface flips that around so that you use existing
systems and iterate them towards the end goal. Just my POV.

FYI, there are other ongoing solutions as well such as existing
ansible tasks directly in the templates today. These are much easier
to reason about when it comes to generating the roles and playbooks,
because it is direct Ansible syntax in the templates, so it's easier
to see the origin of tasks and make changes.

As we move forward on these approaches if we end up with users
gravitating towards certain usage patterns, I think we'd consider
deprecating interfaces that are no longer seen as useful.

>
> Basically, we do this for zuulv2.5 today in openstack-infra (dynamically
> generate playbooks at run-time) and it is a large amount of work to debug
> issues.  In our case, we did it to quickly migrate from jenkins to ansible
> (since zuulv3 completely fixes this with native playbooks) and I wouldn't
> recommend it to operators to do.  Not fun.

I'm not familiar with the technical reasoning there, but on the
surface it sounds similar to what some of our goals may be. We want to
quickly add some Ansible features and move in that direction.

We don't want to write all new roles and playbooks in Ansible, at
least I don't :) We can't even say definitively right now that native
Ansible is the interface we want long term. So I think it would be
premature to approach the problem from the angle of writing new native
playbooks.

Whether or not it ever becomes a full migration is as I said TBD, and
not even something we have to decide now. The decision would be more
driven by how people end up using it.

-- 
-- James Slagle
--