[all] [tc] [osc] [glance] [train] [ptls] Legacy client CLI to OSC review 639376
Hi all, For the Train cycle Artem Goncharov proposed moving the legacy client CLIs to OSC. This goal did not move forward due to a broad range of concerns and was eventually -1'd by Erno Kuvaja (Glance PTL) as he was unable to support it from a Glance point of view. Moving forward, I'd like to end the discussion in the review [1] by abandoning the patch and move this to the mailing list. The following looks like it needs to happen: 1. This PR should be abandoned. It is not going to be accepted as a commmunity goal in this format and the debate within the comments is circular. Let's step out of here, and start having conversation elsewhere. 2. To start those conversations, a pop-up team would be a suitable alternative to begin driving that work. Someone needs to step forward to lead the pop-up team. The TC recommends two individuals step forwards as indicated in the pop-up team Governance document [2]. 3. I recommend the pop-up team include Erno Kuvaja or a representative from the Glance team and including Matt Reidemann (who has been working to close the compute API gaps). 4. The leaders must then identify a clear objective, and a clear disband criteria. If in case the pop-up team decide that this could be better drafted as a community goal for U, then that is a suitable alternative that can be defined using the new goal selection criteria that is being defined here [3] But this should not be decided before clear objectives are defined. Thanks, Alex IRC: asettle [1] https://review.opendev.org/#/c/639376/ [2] https://governance.openstack.org/tc/reference/popup-teams.html [3] https://review.opendev.org/#/c/667932/
Hi everyone, I have abandoned https://review.opendev.org/#/c/639376/ due to the lack of response on the review itself, and this email thread. All discussion regarding the legacy CLI client and OSC to please be constructed through this thread. As part of step 2 of my outlined plan below, I'd be looking for someone with particular interest in the transition of legacy CLI and OSC to step forward and help start a pop-up group. Thank you, Alex On Tue, 2019-08-20 at 08:32 +0000, Alexandra Settle wrote:
Hi all,
For the Train cycle Artem Goncharov proposed moving the legacy client CLIs to OSC. This goal did not move forward due to a broad range of concerns and was eventually -1'd by Erno Kuvaja (Glance PTL) as he was unable to support it from a Glance point of view.
Moving forward, I'd like to end the discussion in the review [1] by abandoning the patch and move this to the mailing list. The following looks like it needs to happen:
1. This PR should be abandoned. It is not going to be accepted as a commmunity goal in this format and the debate within the comments is circular. Let's step out of here, and start having conversation elsewhere.
2. To start those conversations, a pop-up team would be a suitable alternative to begin driving that work. Someone needs to step forward to lead the pop-up team. The TC recommends two individuals step forwards as indicated in the pop-up team Governance document [2].
3. I recommend the pop-up team include Erno Kuvaja or a representative from the Glance team and including Matt Reidemann (who has been working to close the compute API gaps).
4. The leaders must then identify a clear objective, and a clear disband criteria.
If in case the pop-up team decide that this could be better drafted as a community goal for U, then that is a suitable alternative that can be defined using the new goal selection criteria that is being defined here [3] But this should not be decided before clear objectives are defined.
Thanks,
Alex IRC: asettle
[1] https://review.opendev.org/#/c/639376/ [2] https://governance.openstack.org/tc/reference/popup-teams.html [3] https://review.opendev.org/#/c/667932/ -- Alexandra Settle <a.settle@outlook.com> IRC: asettle
On 8/23/2019 6:50 AM, Alexandra Settle wrote:
I have abandonedhttps://review.opendev.org/#/c/639376/ due to the lack of response on the review itself, and this email thread.
All discussion regarding the legacy CLI client and OSC to please be constructed through this thread.
As part of step 2 of my outlined plan below, I'd be looking for someone with particular interest in the transition of legacy CLI and OSC to step forward and help start a pop-up group.
Just FYI, I've submitted a project idea for working on closing compute API feature gaps in this university student/mentor program: http://okrieg.github.io/EC500/index-fall-2019.html I think next steps are the professors compile the submissions and then they get ranked/voted on which students want to work on which projects, so no idea if anyone will want to work on this (nor do I have a public posting of the project idea I submitted since it was just a google form), but if it's selected then we'll have some new contributors to help push the buttons. That doesn't mean I'm signing up for legacy CLI -> OSC pop up SIG WG ballyhoo, just that I did a thing. -- Thanks, Matt
<snip from Alex's original note>
3. I recommend the pop-up team include Erno Kuvaja or a representative from the Glance team and including Matt Reidemann (who has been working to close the compute API gaps).
</snip> Alex Myself or someone from the Cinder team should be involved as well as we have concerns/challenges with removing python-cinderclient. We have been trying to get OSC to parity with the cinderclient CLI but have not been able to get all the way there. Thanks! Jay
4. The leaders must then identify a clear objective, and a clear disband criteria.
If in case the pop-up team decide that this could be better drafted as a community goal for U, then that is a suitable alternative that can be defined using the new goal selection criteria that is being defined here [3] But this should not be decided before clear objectives are defined.
Thanks,
Alex IRC: asettle
[1] https://review.opendev.org/#/c/639376/ [2] https://governance.openstack.org/tc/reference/popup-teams.html [3] https://review.opendev.org/#/c/667932/
On 23/08/2019 15:36, Jay Bryant wrote:
<snip from Alex's original note>
3. I recommend the pop-up team include Erno Kuvaja or a representative from the Glance team and including Matt Reidemann (who has been working to close the compute API gaps).
</snip>
Alex
Myself or someone from the Cinder team should be involved as well as we have concerns/challenges with removing python-cinderclient. We have been trying to get OSC to parity with the cinderclient CLI but have not been able to get all the way there.
Thanks!
Jay
From what I have heard, some of the problems with relying on OSC was reliance on a central (OSC) team for implementation and reviews. Would moving tools like cinder or glance to OSC plugins help solve some of these concerns? This would allow the teams to control the CLI destiny, and even allow teams to add features to team CLIs and OSC simultaneously, while (or if) we transition to OSC.
4. The leaders must then identify a clear objective, and a clear disband criteria.
If in case the pop-up team decide that this could be better drafted as a community goal for U, then that is a suitable alternative that can be defined using the new goal selection criteria that is being defined here [3] But this should not be decided before clear objectives are defined.
Thanks,
Alex IRC: asettle
[1] https://review.opendev.org/#/c/639376/ [2] https://governance.openstack.org/tc/reference/popup-teams.html [3] https://review.opendev.org/#/c/667932/
On 8/23/2019 9:59 AM, Graham Hayes wrote:
From what I have heard, some of the problems with relying on OSC was reliance on a central (OSC) team for implementation and reviews.
Yes, that's true, it's basically Dean doing core reviews these days it seems.
Would moving tools like cinder or glance to OSC plugins help solve some of these concerns? This would allow the teams to control the CLI destiny, and even allow teams to add features to team CLIs and OSC simultaneously, while (or if) we transition to OSC.
This came up at one of the Denver PTGs with Dean in the nova room. The upside is project teams could move things themselves faster, like they do with their own python-*client projects today. They are also the ones that know better how their API works. Nova did this with the osc-placement plugin but still ask Dean for UX guidance from time to time. The downside of project teams owning their parts of the CLI are losing a unified vision for how the commands should be structured which is really one of the main reasons for OSC in the first place, a unified, standardized and good user experience, rather than all of the different little things that each project team did in their CLI. So if we did let project teams own their parts of OSC (I know some already do - placement, ironic, designate, watcher, etc), we would have to trust them to follow whatever guidelines exist since Dean can't be herding all those cats. Honestly I don't really trust (real surprise here right folks?!) most developers (myself included), left alone, to do things the "OSC way" unless they already have some experience with working on OSC - or at least communicating with Dean for UX questions. Based on that, I'd prefer that at least "core" component CLIs remain within OSC and the purview of that core team. But the core team problem needs to be solved which would mean expanding the OSC core team. We definitely need more 'core' project cores (nova/cinder/glance/keystone/neutron) reviewing changes to OSC for their components and maybe that's the path to becoming a core on OSC, if the whole sub-domain core thing works there, I don't know (I'm not on the OSC core team). I would still want Dean overseeing things happening at the component level though, at least until there are more obvious OSC-wide cores, i.e. component core(s) +1/+2 a thing but Dean has final say. Anyway, that's my 2 cents. -- Thanks, Matt
For Octavia, implementing an OSC plugin has worked out exceptionally well. I have also got good feedback from users that OSC is much better than the legacy clients in functionality and ease of use. That said, I have spent some quality time with Dean (thank you again) making sure we didn't stray from the OSC vision. I think like any other guideline or standard in OpenStack we just need to focus on documenting those guidelines and standards well (there are already pretty good documents for this, thank you again to the OSC team). Then it is up to us to pay attention to them and correct our mistakes when we make them. Personally I think all projects should be implemented as plugins. This makes it "obvious" to users when they need a service functionality they need to install the correct plugin. Currently we get "Why doesn't the client support Octavia?" questions when we were one of the early plugins. This also has the advantage of limiting the footprint of the client install for deployments that don't need all of the services. For example, I don't need object nor block storage. It would be nice to not use the disk and tab completion space for those. One of the nice things about the project team owning the plugin is you don't get rogue patches merging that don't align to the project team vision ('-' vs. "_" anyone?). Michael On Fri, Aug 23, 2019 at 1:38 PM Matt Riedemann <mriedemos@gmail.com> wrote:
On 8/23/2019 9:59 AM, Graham Hayes wrote:
From what I have heard, some of the problems with relying on OSC was reliance on a central (OSC) team for implementation and reviews.
Yes, that's true, it's basically Dean doing core reviews these days it seems.
Would moving tools like cinder or glance to OSC plugins help solve some of these concerns? This would allow the teams to control the CLI destiny, and even allow teams to add features to team CLIs and OSC simultaneously, while (or if) we transition to OSC.
This came up at one of the Denver PTGs with Dean in the nova room. The upside is project teams could move things themselves faster, like they do with their own python-*client projects today. They are also the ones that know better how their API works. Nova did this with the osc-placement plugin but still ask Dean for UX guidance from time to time.
The downside of project teams owning their parts of the CLI are losing a unified vision for how the commands should be structured which is really one of the main reasons for OSC in the first place, a unified, standardized and good user experience, rather than all of the different little things that each project team did in their CLI. So if we did let project teams own their parts of OSC (I know some already do - placement, ironic, designate, watcher, etc), we would have to trust them to follow whatever guidelines exist since Dean can't be herding all those cats.
Honestly I don't really trust (real surprise here right folks?!) most developers (myself included), left alone, to do things the "OSC way" unless they already have some experience with working on OSC - or at least communicating with Dean for UX questions. Based on that, I'd prefer that at least "core" component CLIs remain within OSC and the purview of that core team. But the core team problem needs to be solved which would mean expanding the OSC core team. We definitely need more 'core' project cores (nova/cinder/glance/keystone/neutron) reviewing changes to OSC for their components and maybe that's the path to becoming a core on OSC, if the whole sub-domain core thing works there, I don't know (I'm not on the OSC core team). I would still want Dean overseeing things happening at the component level though, at least until there are more obvious OSC-wide cores, i.e. component core(s) +1/+2 a thing but Dean has final say.
Anyway, that's my 2 cents.
--
Thanks,
Matt
On Fri, Aug 23, 2019, at 3:12 PM, Michael Johnson wrote:
For Octavia, implementing an OSC plugin has worked out exceptionally well. I have also got good feedback from users that OSC is much better than the legacy clients in functionality and ease of use.
That said, I have spent some quality time with Dean (thank you again) making sure we didn't stray from the OSC vision.
I think like any other guideline or standard in OpenStack we just need to focus on documenting those guidelines and standards well (there are already pretty good documents for this, thank you again to the OSC team). Then it is up to us to pay attention to them and correct our mistakes when we make them.
Personally I think all projects should be implemented as plugins. This makes it "obvious" to users when they need a service functionality they need to install the correct plugin. Currently we get "Why doesn't the client support Octavia?" questions when we were one of the early plugins. This also has the advantage of limiting the footprint of the client install for deployments that don't need all of the services. For example, I don't need object nor block storage. It would be nice to not use the disk and tab completion space for those. One of the nice things about the project team owning the plugin is you don't get rogue patches merging that don't align to the project team vision ('-' vs. "_" anyone?).
Since you've brought up optimizations here it is worth pointing out that much of OSC's slowness is due to its plugin system. Basically the way python entrypoints work is pkg_resources scans the python path for all packages, sorts them all by name and version, and checks them for entrypoint metadata to know what it can load as an entrypoint. This is slow. Unfortunately that means that the more we get tied to the plugin system the harder it becomes to potentially fix this problem. Personally I'd much rather lose a few megabytes of disk to avoid major startup overhead. That said, the user experience with OSC tends to be much better than that of the python-*clients even when you factor in some slowness. In fact I have to fight very strong urges to give up any time I have to interact with an API that isn't yet supported by OSC.
Michael
reliance on a central (OSC) team for implementation and reviews. <snip>>> Would moving tools like cinder or glance to OSC plugins help solve <snip> upside is project teams could move things themselves faster, like they do with their own python-*client projects today. They are also the ones that know better how their API works. <snip> The downside of project teams owning their parts of the CLI are losing a unified vision <snip> we would have to trust them <snip> I would still want Dean overseeing things happening at the component level though, at least until there are more obvious OSC-wide cores, i.e. component core(s) +1/+2 a thing but Dean has final say.
This is becoming a familiar theme for projects/efforts with touchpoints across OpenStack. Docs deliverables have been moved into individual projects, and "the docs team" is becoming a SIG [1]. But who is the Dean of docs, making sure we retain a common structure, style, and voice across all of these deliverables? SDK is making forays into the "trust component SMEs" arena [2]. The API-SIG is undergoing a similar existential exercise [3], and it is unclear where its deliverables will end up. And this issue with OSC is not new. All of these have their own nuances, but also quite a bit in common. Can we brainstorm ways that might address more than one in a satisfactory way? Here's one idea: Deliverables live in projects under the governance of their component; so for example nova-docs and nova-osc-plugin and nova-sdk-plugin would be under nova governance, with core teams managed by nova. But patches in those projects require an additional vote of a different flavor (a "Rollcall-Vote"?) from the related SIG or core team in order to merge. Thoughts? efried [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.ht... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.ht... [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.ht...
On 8/23/19 5:47 PM, Eric Fried wrote:
reliance on a central (OSC) team for implementation and reviews. <snip>>> Would moving tools like cinder or glance to OSC plugins help solve <snip> upside is project teams could move things themselves faster, like they do with their own python-*client projects today. They are also the ones that know better how their API works. <snip> The downside of project teams owning their parts of the CLI are losing a unified vision <snip> we would have to trust them <snip> I would still want Dean overseeing things happening at the component level though, at least until there are more obvious OSC-wide cores, i.e. component core(s) +1/+2 a thing but Dean has final say.
This is becoming a familiar theme for projects/efforts with touchpoints across OpenStack.
Docs deliverables have been moved into individual projects, and "the docs team" is becoming a SIG [1]. But who is the Dean of docs, making sure we retain a common structure, style, and voice across all of these deliverables?
SDK is making forays into the "trust component SMEs" arena [2].
The API-SIG is undergoing a similar existential exercise [3], and it is unclear where its deliverables will end up.
And this issue with OSC is not new.
All of these have their own nuances, but also quite a bit in common. Can we brainstorm ways that might address more than one in a satisfactory way? Here's one idea:
Deliverables live in projects under the governance of their component; so for example nova-docs and nova-osc-plugin and nova-sdk-plugin would be under nova governance, with core teams managed by nova. But patches in those projects require an additional vote of a different flavor (a "Rollcall-Vote"?) from the related SIG or core team in order to merge.
I don't dislike the idea, but I think the decentralization of these things is at least partially a recognition of the fact that the teams have shrunk to the point where they can't be responsible for reviewing every doc/sdk/cli change across all of openstack (if they ever could in the first place). It may be that the best we can do is have the remaining experts in those areas write up guidelines[0] for the project contributors to follow and then pull the experts in to review specific edge cases and such. The bad thing is that mistakes in stuff like the cli and sdk can be hard to fix once they've been released. I can't tell you how many bugs I ran into because scripts were using the old glance image visibility cli opt that got changed way back when. But that might just be something we have to live with unless I'm wrong about the review capacity of the teams in question. 0: Something like https://docs.openstack.org/python-openstackclient/stein/contributor/humanint... but maybe even more than that since I think I've seen Dean mention best practices that aren't covered in that doc.
Thoughts?
efried
[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.ht... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.ht... [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/thread.ht...
It may be that the best we can do is have the remaining experts in those areas write up guidelines[0] for the project contributors to follow and then pull the experts in to review specific edge cases and such.
If that's the best we can do, so be it. But recent events [1] imply that written guidelines aren't always read/followed /o\
I don't dislike the idea, but I think the decentralization of these things is at least partially a recognition of the fact that the teams have shrunk to the point where they can't be responsible for reviewing every doc/sdk/cli change across all of openstack (if they ever could in the first place).
Right right, the point is to allow them to *just* review the patches from the point of view of conforming to the overall guidelines without feeling responsible for deep technical dives. In other words, reduce the per-patch burden to allow more patches through the pipe. To give a concrete example, if [2] had been in nova-osc-plugin, an OSC core could have vetted it for option/argument naming without sweating the substance of the API interaction, unit tests, etc. So like, a quarter of the patch rather than all of it. efried [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008643.ht... [2] https://review.opendev.org/#/c/675117/ (pretend this was osc rather than novaclient) [3] https://review.opendev.org/#/c/675117/3/novaclient/v2/shell.py
participants (8)
-
Alexandra Settle
-
Ben Nemec
-
Clark Boylan
-
Eric Fried
-
Graham Hayes
-
Jay Bryant
-
Matt Riedemann
-
Michael Johnson