[sdk] R1.0 preparation work
Hey all, As we were discussing during the PTG preparation work for the openstacksdk R1.0 has been started. There is now feature branch “feature/r1” and a set of patches already in place ([1]). (While https://review.opendev.org/c/openstack/devstack/+/791541 <https://review.opendev.org/c/openstack/devstack/+/791541> is not merged there are no functional tests running, but that is not blocking from doing the main work) Things to be done: - get rid of all direct REST calls in the cloud layer. Instead reuse corresponding proxies [must] - generalise tag, metadata, quota(set), limits [must] - clean the code from some deprecated things and py2 remaining (if any) [must] - review resource caching to be implemented on the resource layer [optional] - introduction of read-only resource properties [optional] - restructure documentation to make it more used friendly [optional] Planned R1 interface changes Every cloud layer method (Connection.connection.***, and not i.e. Connection.connection.compute.***) will consistently return Resource objects. At the moment there is a mix of Munch and Resource types depending on the case. Resource class itself fulfil dict, Munch and attribute interface, so there should be no breaking changes for getting info out of it. The only known limitation is that compared to bare dict/Munch it might be not allowed to modify some properties directly on the object. Ansible collection modules [2] would be modified to explicitly convert return of sdk into dict (Resource.to_dict()) to further operate on it. This means that in some cases older Ansible modules (2.7-2.9) will not properly work with newer SDK. Zuul jobs and everybody stuck without possibility to use newer Ansible collection [2] are potential victims here (depends on the real usage pattern). Sadly there is no way around it, since historically ansible modules operate whatever SDK returns and Ansible sometimes decides to alter objects (i.e. no_log case) what we might want to forbid (read-only or virtual properties). Sorry about that. In some rare cases attribute naming (whatever returned by the cloud layer) might be affected (i.e. is_bootable vs bootable). We are going to strictly bring all names to the convention. Due to the big amount of changes touching lot of files here I propose to stop adding new features into the master branch directly and instead put them into feature/r1 branch. I want to keep master during this time more like a stable branch for bugfixes and other important things. Once r1 branch feel ready we will merge it into master and most likely release something like RC to be able to continue integration work with Ansible. Everybody interested in the future of sdk is welcome in doing reviews and code [1] to know what comes and to speed up the work. Any concerns? Suggestions? Regards, Artem [1] https://review.opendev.org/q/project:openstack%252Fopenstacksdk+branch:featu... <https://review.opendev.org/q/project:openstack%2Fopenstacksdk+branch:feature%2Fr1> [2] https://opendev.org/openstack/ansible-collections-openstack <https://opendev.org/openstack/ansible-collections-openstack>
Hi Artem, My one concern about new features only landing in the feature branch is we have dependencies that may require new service project features to land in the main branch. I.e. The Octavia dashboard uses openstacksdk, so if we need to add a new feature we need to land it in SDK at the same time. Can we consider that it is ok for project related new features to land in both branches? Michael On Wed, May 19, 2021 at 3:40 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Hey all,
As we were discussing during the PTG preparation work for the openstacksdk R1.0 has been started. There is now feature branch “feature/r1” and a set of patches already in place ([1]). (While https://review.opendev.org/c/openstack/devstack/+/791541 is not merged there are no functional tests running, but that is not blocking from doing the main work)
Things to be done:
- get rid of all direct REST calls in the cloud layer. Instead reuse corresponding proxies [must] - generalise tag, metadata, quota(set), limits [must] - clean the code from some deprecated things and py2 remaining (if any) [must] - review resource caching to be implemented on the resource layer [optional] - introduction of read-only resource properties [optional] - restructure documentation to make it more used friendly [optional]
Planned R1 interface changes
Every cloud layer method (Connection.connection.***, and not i.e. Connection.connection.compute.***) will consistently return Resource objects. At the moment there is a mix of Munch and Resource types depending on the case. Resource class itself fulfil dict, Munch and attribute interface, so there should be no breaking changes for getting info out of it. The only known limitation is that compared to bare dict/Munch it might be not allowed to modify some properties directly on the object. Ansible collection modules [2] would be modified to explicitly convert return of sdk into dict (Resource.to_dict()) to further operate on it. This means that in some cases older Ansible modules (2.7-2.9) will not properly work with newer SDK. Zuul jobs and everybody stuck without possibility to use newer Ansible collection [2] are potential victims here (depends on the real usage pattern). Sadly there is no way around it, since historically ansible modules operate whatever SDK returns and Ansible sometimes decides to alter objects (i.e. no_log case) what we might want to forbid (read-only or virtual properties). Sorry about that. In some rare cases attribute naming (whatever returned by the cloud layer) might be affected (i.e. is_bootable vs bootable). We are going to strictly bring all names to the convention.
Due to the big amount of changes touching lot of files here I propose to stop adding new features into the master branch directly and instead put them into feature/r1 branch. I want to keep master during this time more like a stable branch for bugfixes and other important things. Once r1 branch feel ready we will merge it into master and most likely release something like RC to be able to continue integration work with Ansible.
Everybody interested in the future of sdk is welcome in doing reviews and code [1] to know what comes and to speed up the work.
Any concerns? Suggestions?
Regards, Artem
[1] https://review.opendev.org/q/project:openstack%252Fopenstacksdk+branch:featu... [2] https://opendev.org/openstack/ansible-collections-openstack
Yes, absolutely. I would prefer us to implement those in feature branch and cherry-picking those to master.
On 20. May 2021, at 18:34, Michael Johnson <johnsomor@gmail.com> wrote:
Hi Artem,
My one concern about new features only landing in the feature branch is we have dependencies that may require new service project features to land in the main branch. I.e. The Octavia dashboard uses openstacksdk, so if we need to add a new feature we need to land it in SDK at the same time.
Can we consider that it is ok for project related new features to land in both branches?
Michael
On Wed, May 19, 2021 at 3:40 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Hey all,
As we were discussing during the PTG preparation work for the openstacksdk R1.0 has been started. There is now feature branch “feature/r1” and a set of patches already in place ([1]). (While https://review.opendev.org/c/openstack/devstack/+/791541 is not merged there are no functional tests running, but that is not blocking from doing the main work)
Things to be done:
- get rid of all direct REST calls in the cloud layer. Instead reuse corresponding proxies [must] - generalise tag, metadata, quota(set), limits [must] - clean the code from some deprecated things and py2 remaining (if any) [must] - review resource caching to be implemented on the resource layer [optional] - introduction of read-only resource properties [optional] - restructure documentation to make it more used friendly [optional]
Planned R1 interface changes
Every cloud layer method (Connection.connection.***, and not i.e. Connection.connection.compute.***) will consistently return Resource objects. At the moment there is a mix of Munch and Resource types depending on the case. Resource class itself fulfil dict, Munch and attribute interface, so there should be no breaking changes for getting info out of it. The only known limitation is that compared to bare dict/Munch it might be not allowed to modify some properties directly on the object. Ansible collection modules [2] would be modified to explicitly convert return of sdk into dict (Resource.to_dict()) to further operate on it. This means that in some cases older Ansible modules (2.7-2.9) will not properly work with newer SDK. Zuul jobs and everybody stuck without possibility to use newer Ansible collection [2] are potential victims here (depends on the real usage pattern). Sadly there is no way around it, since historically ansible modules operate whatever SDK returns and Ansible sometimes decides to alter objects (i.e. no_log case) what we might want to forbid (read-only or virtual properties). Sorry about that. In some rare cases attribute naming (whatever returned by the cloud layer) might be affected (i.e. is_bootable vs bootable). We are going to strictly bring all names to the convention.
Due to the big amount of changes touching lot of files here I propose to stop adding new features into the master branch directly and instead put them into feature/r1 branch. I want to keep master during this time more like a stable branch for bugfixes and other important things. Once r1 branch feel ready we will merge it into master and most likely release something like RC to be able to continue integration work with Ansible.
Everybody interested in the future of sdk is welcome in doing reviews and code [1] to know what comes and to speed up the work.
Any concerns? Suggestions?
Regards, Artem
[1] https://review.opendev.org/q/project:openstack%252Fopenstacksdk+branch:featu... [2] https://opendev.org/openstack/ansible-collections-openstack
Great, I will try to make sure anyone that needs to add a feature for Octavia/Designate will follow that process. Michael On Thu, May 20, 2021 at 9:36 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Yes, absolutely. I would prefer us to implement those in feature branch and cherry-picking those to master.
On 20. May 2021, at 18:34, Michael Johnson <johnsomor@gmail.com> wrote:
Hi Artem,
My one concern about new features only landing in the feature branch is we have dependencies that may require new service project features to land in the main branch. I.e. The Octavia dashboard uses openstacksdk, so if we need to add a new feature we need to land it in SDK at the same time.
Can we consider that it is ok for project related new features to land in both branches?
Michael
On Wed, May 19, 2021 at 3:40 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Hey all,
As we were discussing during the PTG preparation work for the openstacksdk R1.0 has been started. There is now feature branch “feature/r1” and a set of patches already in place ([1]). (While https://review.opendev.org/c/openstack/devstack/+/791541 is not merged there are no functional tests running, but that is not blocking from doing the main work)
Things to be done:
- get rid of all direct REST calls in the cloud layer. Instead reuse corresponding proxies [must] - generalise tag, metadata, quota(set), limits [must] - clean the code from some deprecated things and py2 remaining (if any) [must] - review resource caching to be implemented on the resource layer [optional] - introduction of read-only resource properties [optional] - restructure documentation to make it more used friendly [optional]
Planned R1 interface changes
Every cloud layer method (Connection.connection.***, and not i.e. Connection.connection.compute.***) will consistently return Resource objects. At the moment there is a mix of Munch and Resource types depending on the case. Resource class itself fulfil dict, Munch and attribute interface, so there should be no breaking changes for getting info out of it. The only known limitation is that compared to bare dict/Munch it might be not allowed to modify some properties directly on the object. Ansible collection modules [2] would be modified to explicitly convert return of sdk into dict (Resource.to_dict()) to further operate on it. This means that in some cases older Ansible modules (2.7-2.9) will not properly work with newer SDK. Zuul jobs and everybody stuck without possibility to use newer Ansible collection [2] are potential victims here (depends on the real usage pattern). Sadly there is no way around it, since historically ansible modules operate whatever SDK returns and Ansible sometimes decides to alter objects (i.e. no_log case) what we might want to forbid (read-only or virtual properties). Sorry about that. In some rare cases attribute naming (whatever returned by the cloud layer) might be affected (i.e. is_bootable vs bootable). We are going to strictly bring all names to the convention.
Due to the big amount of changes touching lot of files here I propose to stop adding new features into the master branch directly and instead put them into feature/r1 branch. I want to keep master during this time more like a stable branch for bugfixes and other important things. Once r1 branch feel ready we will merge it into master and most likely release something like RC to be able to continue integration work with Ansible.
Everybody interested in the future of sdk is welcome in doing reviews and code [1] to know what comes and to speed up the work.
Any concerns? Suggestions?
Regards, Artem
[1] https://review.opendev.org/q/project:openstack%252Fopenstacksdk+branch:featu... [2] https://opendev.org/openstack/ansible-collections-openstack
On Thu, May 20, 2021 at 9:39 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Yes, absolutely. I would prefer us to implement those in feature branch and cherry-picking those to master.
+1 Awesome. We have a number of manila patches in-flight that I wish to refresh, i'll retarget them to the r1 branch and cherry-pick into master when they merge. We don't yet use openstacksdk in manila-ui or python-manilaclient, or have our own ansible collections modules yet - so while we chase features, this may be a delayed concern for the manila folks.
On 20. May 2021, at 18:34, Michael Johnson <johnsomor@gmail.com> wrote:
Hi Artem,
My one concern about new features only landing in the feature branch is we have dependencies that may require new service project features to land in the main branch. I.e. The Octavia dashboard uses openstacksdk, so if we need to add a new feature we need to land it in SDK at the same time.
Can we consider that it is ok for project related new features to land in both branches?
Michael
On Wed, May 19, 2021 at 3:40 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Hey all,
As we were discussing during the PTG preparation work for the
openstacksdk R1.0 has been started. There is now feature branch “feature/r1” and a set of patches already in place ([1]). (While https://review.opendev.org/c/openstack/devstack/+/791541 is not merged there are no functional tests running, but that is not blocking from doing the main work)
Things to be done:
- get rid of all direct REST calls in the cloud layer. Instead reuse
corresponding proxies [must]
- generalise tag, metadata, quota(set), limits [must] - clean the code from some deprecated things and py2 remaining (if any) [must] - review resource caching to be implemented on the resource layer [optional] - introduction of read-only resource properties [optional] - restructure documentation to make it more used friendly [optional]
Planned R1 interface changes
Every cloud layer method (Connection.connection.***, and not i.e. Connection.connection.compute.***) will consistently return Resource objects. At the moment there is a mix of Munch and Resource types depending on the case. Resource class itself fulfil dict, Munch and attribute interface, so there should be no breaking changes for getting info out of it. The only known limitation is that compared to bare dict/Munch it might be not allowed to modify some properties directly on the object. Ansible
collection modules [2] would be modified to explicitly convert return of sdk into dict (Resource.to_dict()) to further operate on it. This means that in some cases older Ansible modules (2.7-2.9) will not properly work with newer SDK. Zuul jobs and everybody stuck without possibility to use newer Ansible collection [2] are potential victims here (depends on the real usage pattern). Sadly there is no way around it, since historically ansible modules operate whatever SDK returns and Ansible sometimes decides to alter objects (i.e. no_log case) what we might want to forbid (read-only or virtual properties). Sorry about that. >> In some rare cases attribute naming (whatever returned by the cloud layer) might be affected (i.e. is_bootable vs bootable). We are going to strictly bring all names to the convention. >> >> >> Due to the big amount of changes touching lot of files here I propose to stop adding new features into the master branch directly and instead put them into feature/r1 branch. I want to keep master during this time more like a stable branch for bugfixes and other important things. Once r1 branch feel ready we will merge it into master and most likely release something like RC to be able to continue integration work with Ansible. >> >> >> Everybody interested in the future of sdk is welcome in doing reviews and code [1] to know what comes and to speed up the work. >> >> >> Any concerns? Suggestions? >> >> >> Regards, >> Artem >> >> >> [1] https://review.opendev.org/q/project:openstack%252Fopenstacksdk+branch:feature%252Fr1 >> [2] https://opendev.org/openstack/ansible-collections-openstack >>
Awesome. My rough plan is to have feature branch merged back to master by end of summer (but you know - all depends on reviews speed). Artem
On 20. May 2021, at 18:34, Michael Johnson <johnsomor@gmail.com> wrote:
Hi Artem,
My one concern about new features only landing in the feature branch is we have dependencies that may require new service project features to land in the main branch. I.e. The Octavia dashboard uses openstacksdk, so if we need to add a new feature we need to land it in SDK at the same time.
Can we consider that it is ok for project related new features to land in both branches?
Michael
On Wed, May 19, 2021 at 3:40 AM Artem Goncharov <artem.goncharov@gmail.com> wrote:
Hey all,
As we were discussing during the PTG preparation work for the openstacksdk R1.0 has been started. There is now feature branch “feature/r1” and a set of patches already in place ([1]). (While https://review.opendev.org/c/openstack/devstack/+/791541 is not merged there are no functional tests running, but that is not blocking from doing the main work)
Things to be done:
- get rid of all direct REST calls in the cloud layer. Instead reuse corresponding proxies [must] - generalise tag, metadata, quota(set), limits [must] - clean the code from some deprecated things and py2 remaining (if any) [must] - review resource caching to be implemented on the resource layer [optional] - introduction of read-only resource properties [optional] - restructure documentation to make it more used friendly [optional]
Planned R1 interface changes
Every cloud layer method (Connection.connection.***, and not i.e. Connection.connection.compute.***) will consistently return Resource objects. At the moment there is a mix of Munch and Resource types depending on the case. Resource class itself fulfil dict, Munch and attribute interface, so there should be no breaking changes for getting info out of it. The only known limitation is that compared to bare dict/Munch it might be not allowed to modify some properties directly on the object. Ansible collection modules [2] would be modified to explicitly convert return of sdk into dict (Resource.to_dict()) to further operate on it. This means that in some cases older Ansible modules (2.7-2.9) will not properly work with newer SDK. Zuul jobs and everybody stuck without possibility to use newer Ansible collection [2] are potential victims here (depends on the real usage pattern). Sadly there is no way around it, since historically ansible modules operate whatever SDK returns and Ansible sometimes decides to alter objects (i.e. no_log case) what we might want to forbid (read-only or virtual properties). Sorry about that. In some rare cases attribute naming (whatever returned by the cloud layer) might be affected (i.e. is_bootable vs bootable). We are going to strictly bring all names to the convention.
Due to the big amount of changes touching lot of files here I propose to stop adding new features into the master branch directly and instead put them into feature/r1 branch. I want to keep master during this time more like a stable branch for bugfixes and other important things. Once r1 branch feel ready we will merge it into master and most likely release something like RC to be able to continue integration work with Ansible.
Everybody interested in the future of sdk is welcome in doing reviews and code [1] to know what comes and to speed up the work.
Any concerns? Suggestions?
Regards, Artem
[1] https://review.opendev.org/q/project:openstack%252Fopenstacksdk+branch:featu... [2] https://opendev.org/openstack/ansible-collections-openstack
participants (3)
-
Artem Goncharov
-
Goutham Pacha Ravi
-
Michael Johnson