Hi,

I feel that this might be a good topic to discuss in terms of ansible SIG [1] and it can become the new home. So maybe we can plan a meeting for closer and prodictive discussion?

[1] https://etherpad.openstack.org/p/ansible-sig

28.11.2019, 11:45, "Mark Goddard" mark@stackhpc.com:

On Wed, 27 Nov 2019 at 17:58, Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all

in the light of finding the new home place for openstack related ansible modules [1] I'd like to discuss the best strategy to create Ironic ansible modules. Existing Ironic modules in Ansible repo don't cover even half of Ironic functionality, don't fit current needs and definitely require an additional work. There are a few topics that require attention and better be solved before modules are written to save additional work. We prepared an etherpad [2] with all these questions and if you have ideas or suggestions on how it should look you're welcome to update it.  We'd like to decide the final place for them, name conventions (the most complex one!), what they should look like and how better to implement.  Anybody interested in Ansible and baremetal management in Openstack, you're more than welcome to contribute.

Thanks for raising this, we're definitely missing some key things for ironic. I added a couple of roles and modules that we developed for kayobe to the etherpad. Would be happy to contribute them to the collection.

...

Thanks

[1] https://review.opendev.org/#/c/684740/  [2] https://etherpad.openstack.org/p/ironic-ansible-modules

--  Best regards  Sagi Shnaidman

--  Kind Regards, Dmitriy Rabotyagov

Hi

On 28. Nov 2019, at 13:47, Dmitry Tantsur dtantsur@redhat.com wrote:

Hi,

On Thu, Nov 28, 2019 at 1:07 PM Dmitriy Rabotyagov <noonedeadpunk@ya.ru mailto:noonedeadpunk@ya.ru> wrote: Hi,

I feel that this might be a good topic to discuss in terms of ansible SIG [1] and it can become the new home. So maybe we can plan a meeting for closer and prodictive discussion?

I think it's a great idea. Do you have any formal meetings yet or should we maybe schedule a separate one?

Dmitry

There is a bigger issue to be resolved first: what is the home for OS Ansible modules (https://review.opendev.org/#/q/project:openstack/ansible-collections-opensta... https://review.opendev.org/#/q/project:openstack/ansible-collections-openstack, started by Monty recently. Somehow I struggle to find original discussion on that)? How those are treated now part of Ansible now doesn’t really look that much promising/effective and I have just a big bunch of improvements waiting for OS to overtake development.

Let's choose an appropriate time in the etherpad [1], starting from Tue next week to avoid a short notice. I'll send an update with exact time in Sunday 01 Dec. Thanks

[1] https://etherpad.openstack.org/p/ironic-ansible-modules

On Thu, Nov 28, 2019 at 5:48 PM Dmitriy Rabotyagov noonedeadpunk@ya.ru wrote:

We have SIG meetings pretty ocasionally nowadays and it was a while since the last one. But we used to held them on Firdays at 2pm UTC in #openstack-ansible-sig IRC channel.

28.11.2019, 14:47, "Dmitry Tantsur" dtantsur@redhat.com:

Hi,

On Thu, Nov 28, 2019 at 1:07 PM Dmitriy Rabotyagov noonedeadpunk@ya.ru wrote:

Hi,

I feel that this might be a good topic to discuss in terms of ansible SIG [1] and it can become the new home. So maybe we can plan a meeting for closer and prodictive discussion?

I think it's a great idea. Do you have any formal meetings yet or should we maybe schedule a separate one?

Dmitry

[1] https://etherpad.openstack.org/p/ansible-sig

28.11.2019, 11:45, "Mark Goddard" mark@stackhpc.com:

...

On Wed, 27 Nov 2019 at 17:58, Sagi Shnaidman sshnaidm@redhat.com

wrote:

...

...

Hi, all

in the light of finding the new home place for openstack related

ansible modules [1] I'd like to discuss the best strategy to create Ironic ansible modules. Existing Ironic modules in Ansible repo don't cover even half of Ironic functionality, don't fit current needs and definitely require an additional work. There are a few topics that require attention and better be solved before modules are written to save additional work. We prepared an etherpad [2] with all these questions and if you have ideas or suggestions on how it should look you're welcome to update it.

...

...

We'd like to decide the final place for them, name conventions (the

most complex one!), what they should look like and how better to implement.

...

...

Anybody interested in Ansible and baremetal management in Openstack,

you're more than welcome to contribute.

...

Thanks for raising this, we're definitely missing some key things for ironic. I added a couple of roles and modules that we developed for kayobe to the etherpad. Would be happy to contribute them to the collection.

...

Thanks

[1] https://review.opendev.org/#/c/684740/ [2] https://etherpad.openstack.org/p/ironic-ansible-modules

-- Best regards Sagi Shnaidman

-- Kind Regards, Dmitriy Rabotyagov

-- Kind Regards, Dmitriy Rabotyagov

On Wed, 2019-12-04 at 12:43 +0000, Jesse Pretorius wrote:

I can't seem to find the source for https://governance.openstack.org/sigs/ - if someone can let me know where that is then I'll push up a patch to point it at the wiki. I'd be happy to update the wiki page too with the content from the etherpad, assuming that the content is deemed appropriate.

https://opendev.org/openstack/governance-sigs . Though if possible the information should be in governance. Thanks for updating things!

JP

On Wed, 2019-12-04 at 13:22 -0500, Paul Belanger wrote:

Could we also update http://eavesdrop.openstack.org/ to reference the ansible-sig meeting time? I'd like to add this info into calander.

It is discussed here [1].

Regards, JP

[1]: https://review.opendev.org/#/c/697278/

Hi, all short minutes from the meeting today about moving of Openstack Ansible modules to Openstack.

1. Because of some level of uncertainty and different opinions, the details of treatment of old modules will be under discussion in ML. I'll send a mail about this topic. 2. We agreed to have modules under "openstack." namespace and named "cloud". So regular modules will be named like "openstack.cloud.os_server" for example. 3. We agreed to keep Ansible modules as thin as possible, putting the logic into SDK. 4. Also we will keep compatibility with as much Ansible versions as possible. 5. We agreed to have manual releases of Ansible modules as much as we need. Similarly as it's done with SDK.

Logs: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.0... Minutes: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.0... Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules

Next time: Thursday 19 Dec 2019 4.00 PM UTC.

Thanks

On Fri, Dec 6, 2019 at 12:03 AM Sagi Shnaidman sshnaidm@redhat.com wrote:

Hi, all short minutes from the meeting today about Openstack Ansible modules.

1. Ansible 2.10 is going to move all modules to collections, so Openstack

modules should find a new home in Openstack repos. 2. Namespace for openstack modules will be named "openstack.". What is coming after the dot is still under discussion. 3. Current modules will be migrated to collections in "openstack." as is with their names and will be still available for playbooks (via symlinking). It will avoid breaking people that use in their playbooks os_* modules now. 4. Old modules will be frozen after migrations and all development work will go in the new modules which will live aside. 5. Critical bugfixes to 2.9 versions will be done via Ansible GitHub repo as usual and synced manually to "openstack." collection. It must be a very exceptional case. 6. Migrations are set for mid of January 2020 approximately. 7. Modules should stay compatible with last Ansible and collections API changed. 8. Because current old modules are licensed with GPL and license of Openstack is Apache2, we need to figure out if we can either relicense them or develop new ones with different license or to continue to work on new ones with GPL in SIG repo. Agreed to ask on legal-discuss ML.

Long minutes: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.0... Logs: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.0...

Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules Next time Thursday 12 Dec 2019 4.00 PM UTC.

Thanks

On Tue, Dec 3, 2019 at 8:18 PM Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all In the meeting today we agreed to meet every Thursday starting *this week* at 4.00 PM UTC on #openstack-sdks channel on Freenode. We'll discuss everything related to Openstack Ansible modules. Agenda and topics are in the etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules (I've created a new one, because we don't limit to Ironic modules only, it's about all of them in general)

Short minutes from meeting today: Organizational:

1. We meet every Thursday from this week at 4.00 PM UTC on #openstack-sdks
2. Interested parties for now are: Ironic, Tripleo, Openstack-Ansible,

Kolla-ansible, OpenstackSDK teams. Feel free to join and add yourself in the etherpad. [1] 3. We'll track our work in Storyboard for ansible-collections-openstack (in progress) 4. Openstack Ansible modules will live as collections under Ansible SIG in repo openstack/ansible-collections-openstack [2] because there are issues with different licensing: GPLv3 for Ansible in upstream and Openstack license (Apache2). 5. Ansible upstream Openstack modules will be merge-frozen when we'll have our collections fully working and will be deprecated from Ansible at some point in the future. 6. Openstack Ansible collections will be published to Galaxy. 7. There is a list of people that can be pinged for reviews in ansible-collections-openstack project, feel free to join there [1]

Technical:

1. We use openstacksdk instead of [project]client modules.
2. We will rename modules to be more like os_[service_type] named,

examples are in Ironic modules etherpad [3]

Logs from meeting today you can find here: http://eavesdrop.openstack.org/meetings/ansible_sig/2019/ansible_sig.2019-12... Please feel free to participate and add topics to agenda. [1]

[1] https://etherpad.openstack.org/p/openstack-ansible-modules [2] https://review.opendev.org/#/c/684740/ [3] https://etherpad.openstack.org/p/ironic-ansible-modules

Thanks

On Wed, Nov 27, 2019 at 7:57 PM Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all

in the light of finding the new home place for openstack related ansible modules [1] I'd like to discuss the best strategy to create Ironic ansible modules. Existing Ironic modules in Ansible repo don't cover even half of Ironic functionality, don't fit current needs and definitely require an additional work. There are a few topics that require attention and better be solved before modules are written to save additional work. We prepared an etherpad [2] with all these questions and if you have ideas or suggestions on how it should look you're welcome to update it. We'd like to decide the final place for them, name conventions (the most complex one!), what they should look like and how better to implement. Anybody interested in Ansible and baremetal management in Openstack, you're more than welcome to contribute.

Thanks

[1] https://review.opendev.org/#/c/684740/ [2] https://etherpad.openstack.org/p/ironic-ansible-modules

-- Best regards Sagi Shnaidman

-- Best regards Sagi Shnaidman

-- Best regards Sagi Shnaidman

On Dec 16, 2019, at 11:46 AM, Sagi Shnaidman sshnaidm@redhat.com wrote:

Hi, all

recently we had an Openstack Ansible modules meeting, when were discussing how we will manage current modules and new planned modules. Because it was a hot topic which took most of the time and no agreement was reached, I think it's worth to discuss it here in ML.

Options that have been raised in the discussion[1]:

1. To freeze current modules and start writing new modules from scratch
2. To freeze current modules and based on them develop new modules
3. To continue to work on current modules and change them step by step

I believe we should, generally speaking, do this. The ansible folks have some things in place (or plan to have some things in place) that should make this reorg otherwise transparent, so I don’t think we should just change things on users.

I followed up with gundalow about how things are going to work - here is a summary:

In ansible/ansible there is a file called BOTMETA.yml. In that file, we’ll put in entries like:

$modules/cloud/openstack/openstack/os_server: migrated_to: openstack.cloud.server

This lets us map old module names `os_server` to new collection paths `openstack.cloud.server`.

When the ansible source tarball or wheel are made, BOTMETA will be consulted, needed collections will be downloaded and symlinks will be made from the old module names to the new module names.

So - for a user who runs “pip install ansible” (and presumably also dnf install ansible or apt-get install ansible) - they will get things installed as they always have been and they will not need to break any of their playbooks.

This is intended to be done in perpetuity.

The lifecycle of our new modules n the collection is a continuation of the existing modules - so I believe we should take care to maintain backwards compat.

Also - since someone asked at the meeting - yes, this includes things like the dynamic inventory plugin - so we’ll be moving all of the content over.

It is currently undefined what the behavior will be if someone says “pip install ansible ; ansible-galaxy collection install openstack.cloud” For sure doing that and then referencing openstack.cloud.server in a playbook will get the new content. It is undefined as of now whether that will update os_server. Toshio is working on defining this, so if we have opinions we should talk to him about it.

Because we have the renaming capability via BOTMETA - we get a nice opportunity to clean things up in our collection. For that reason, I believe we are all agreed to:

* Drop os_ underscore in the collection version * Drop service_type prefix on ‘admin” modules

So that will have: * os_server -> oppenstack.cloud.server * os_ironic_node -> openstack.cloud.baremetal_node (or something better)

NOW - there are some modules we may need to be more aggressive with as far as changes go. I think for those, we have a few options:

- Do the breaking changes behind a feature flag in the parameters, set a deprecation on the old behavior and after 2 releases change the default on the feature flag. - Make a completely new module and deprecate the old one.

But I think we should take each case of that as it comes as a specific case. The one thing that has been brought up specifically is Dmitry mentioned we may need to improve the auth story for standalone ironic. That may be true - but I’m not yet convinced this needs to be breaking. Yet. But if it needs to be - we can solve it.

Because we have control over the whole collection - we can do a better job with module_utils - and we can make additional constructor methods if needed. I’ve got a patch up:

https://review.opendev.org/#/c/698044/

With an example of making a base class instead of a factory function. It might not yet be the right design, but hopefully it shows how we can clean some things up as we move forward.

4. In case of freezing current modules, deprecate them later

Things to consider:

1. People are using current modules in playbooks and we don't want to break them, so current modules interfaces should stay available and not change for not breaking backward compatibility
2. We might redesign some of modules, including big changes in common modules parts
3. We might redistribute current module functionality over other and new modules

I think it can be a start for the discussion on how to move further, please comment.

Thanks

[1] http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.0...

Hi, all

I'm happy to inform you that the transition has started.

1. Transition steps are tracked in the etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules 2. Modules have been moved with ansible tool migrate.py according to Ansible guidelines. 3. The job that does linting with ansible-test was created and added to collections repo. Ansible-test sanity is used in Ansible CI for all modules, I think it's worth to have it enabled for us too. As well as pep8 checks. 4. I've ported job that was running on Ansible Github PRs as CI job to be check job in collections repo. It will give us at least the same coverage we had before. Of course, it's a temporary solution that should be revisited and most likely replaced by different kinds of jobs later. And with much more test coverage that it has now. So please review and merge: https://review.opendev.org/#/c/703383/ Any noncritical issues can be fixed in following up patches, let's unblock work on collections, please. 5. The current CI job from SDK repo that was running on Ansible Github patches is changed to print a message about moving modules and fail, not to allow to merge new patches easily. The patch is https://review.opendev.org/#/c/703342/ Instead of it, the new ported job from the previous point will run on SDK repo to ensure we don't break modules with SDK patches. 6. As was agreed before we'll need to rename all modules, link to new names in BOTMETA in Ansible and remove modules from Ansible repo. Renaming is a big chunk of work, so I'd appreciate any help here. Of course, any work can be merged only after CI job will be merged and start run on patches: https://review.opendev.org/#/c/703383/ 7. I posted the warning message on all PRs in Ansible Github that has a relation to Openstack modules, using "openstack" modules. Hopefully, it didn't miss anything. 8. Warning mail will be sent today to Ansible dev Google Group about the transition. 9. All discussions are in channel: #openstack-ansible-sig on Freenode. 10. Please help with reviews and merges as we want to do it asap to prevent big diverges between modules in Ansible and Openstack, also not to block people for a long time.

Thanks

On Tue, Jan 7, 2020 at 1:20 PM Sagi Shnaidman sshnaidm@redhat.com wrote:

Hi, last meeting was pretty short and only 2 participants due to holidays, so I think we can discuss this week the same agenda. I remind that we agreed to move ansible modules after 13 January.

• what is the best strategy for freezing current modules in Ansible?

Because a few patches were merged just recently [1] Seems like "freezing" doesn't really work.

• python versions support in modules
• keeping history when moving modules

and other topics [2] Please add your questions to "Open discussion" section if there are some. Thanks

[1] https://github.com/ansible/ansible/commits/devel/lib/ansible/modules/cloud/o... [2] https://etherpad.openstack.org/p/openstack-ansible-modules

On Fri, Dec 13, 2019 at 12:00 AM Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all short minutes from the meeting today about moving of Openstack Ansible modules to Openstack.

1. Because of some level of uncertainty and different opinions, the

details of treatment of old modules will be under discussion in ML. I'll send a mail about this topic. 2. We agreed to have modules under "openstack." namespace and named "cloud". So regular modules will be named like "openstack.cloud.os_server" for example. 3. We agreed to keep Ansible modules as thin as possible, putting the logic into SDK. 4. Also we will keep compatibility with as much Ansible versions as possible. 5. We agreed to have manual releases of Ansible modules as much as we need. Similarly as it's done with SDK.

Logs: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.0... Minutes: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.0... Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules

Next time: Thursday 19 Dec 2019 4.00 PM UTC.

Thanks

On Fri, Dec 6, 2019 at 12:03 AM Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all short minutes from the meeting today about Openstack Ansible modules.

1. Ansible 2.10 is going to move all modules to collections, so

Openstack modules should find a new home in Openstack repos. 2. Namespace for openstack modules will be named "openstack.". What is coming after the dot is still under discussion. 3. Current modules will be migrated to collections in "openstack." as is with their names and will be still available for playbooks (via symlinking). It will avoid breaking people that use in their playbooks os_* modules now. 4. Old modules will be frozen after migrations and all development work will go in the new modules which will live aside. 5. Critical bugfixes to 2.9 versions will be done via Ansible GitHub repo as usual and synced manually to "openstack." collection. It must be a very exceptional case. 6. Migrations are set for mid of January 2020 approximately. 7. Modules should stay compatible with last Ansible and collections API changed. 8. Because current old modules are licensed with GPL and license of Openstack is Apache2, we need to figure out if we can either relicense them or develop new ones with different license or to continue to work on new ones with GPL in SIG repo. Agreed to ask on legal-discuss ML.

Long minutes: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.0... Logs: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.0...

Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules Next time Thursday 12 Dec 2019 4.00 PM UTC.

Thanks

On Tue, Dec 3, 2019 at 8:18 PM Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all In the meeting today we agreed to meet every Thursday starting *this week* at 4.00 PM UTC on #openstack-sdks channel on Freenode. We'll discuss everything related to Openstack Ansible modules. Agenda and topics are in the etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules (I've created a new one, because we don't limit to Ironic modules only, it's about all of them in general)

Short minutes from meeting today: Organizational:

1. We meet every Thursday from this week at 4.00 PM UTC on

#openstack-sdks 2. Interested parties for now are: Ironic, Tripleo, Openstack-Ansible, Kolla-ansible, OpenstackSDK teams. Feel free to join and add yourself in the etherpad. [1] 3. We'll track our work in Storyboard for ansible-collections-openstack (in progress) 4. Openstack Ansible modules will live as collections under Ansible SIG in repo openstack/ansible-collections-openstack [2] because there are issues with different licensing: GPLv3 for Ansible in upstream and Openstack license (Apache2). 5. Ansible upstream Openstack modules will be merge-frozen when we'll have our collections fully working and will be deprecated from Ansible at some point in the future. 6. Openstack Ansible collections will be published to Galaxy. 7. There is a list of people that can be pinged for reviews in ansible-collections-openstack project, feel free to join there [1]

Technical:

1. We use openstacksdk instead of [project]client modules.
2. We will rename modules to be more like os_[service_type] named,

examples are in Ironic modules etherpad [3]

Logs from meeting today you can find here: http://eavesdrop.openstack.org/meetings/ansible_sig/2019/ansible_sig.2019-12... Please feel free to participate and add topics to agenda. [1]

[1] https://etherpad.openstack.org/p/openstack-ansible-modules [2] https://review.opendev.org/#/c/684740/ [3] https://etherpad.openstack.org/p/ironic-ansible-modules

Thanks

On Wed, Nov 27, 2019 at 7:57 PM Sagi Shnaidman sshnaidm@redhat.com wrote:

...

Hi, all

in the light of finding the new home place for openstack related ansible modules [1] I'd like to discuss the best strategy to create Ironic ansible modules. Existing Ironic modules in Ansible repo don't cover even half of Ironic functionality, don't fit current needs and definitely require an additional work. There are a few topics that require attention and better be solved before modules are written to save additional work. We prepared an etherpad [2] with all these questions and if you have ideas or suggestions on how it should look you're welcome to update it. We'd like to decide the final place for them, name conventions (the most complex one!), what they should look like and how better to implement. Anybody interested in Ansible and baremetal management in Openstack, you're more than welcome to contribute.

Thanks

[1] https://review.opendev.org/#/c/684740/ [2] https://etherpad.openstack.org/p/ironic-ansible-modules

-- Best regards Sagi Shnaidman

-- Best regards Sagi Shnaidman

-- Best regards Sagi Shnaidman

-- Best regards Sagi Shnaidman

-- Best regards Sagi Shnaidman