[dev][requirements][tripleo] Return of the revenge of lockfile strikes back part II
It became apparent in a Python community discussion[0] yesterday that lockfile has been designated as a "critical project" by PyPI, even though it was effectively abandoned in 2010. Because our release automation account is listed as one of its maintainers, I looked into whether we still need it for anything... For those who have been around here a while, you may recall that the OpenStack project temporarily assumed maintenance[1] of lockfile in late 2014 and uploaded a few new releases for it, as a stop-gap until we could replace our uses with oslo.concurrency. That work was completed[2] early in the Liberty development cycle. Unfortunately for us, that's not the end of the story. I looked yesterday expecting to see that we've not needed lockfile for 7 years, and was disappointed to discover it's still in our constraints list. Why? After much wailing and gnashing of teeth and manually installing multiple bisections of the requirements list, I narrowed it down to one dependency: ansible-runner. Apparently, ansible-runner currently depends[3] on python-daemon, which still has a dependency on lockfile[4]. Our uses of ansible-runner seem to be pretty much limited to TripleO repositories (hence tagging them in the subject), so it's possible they could find an alternative to it and solve this dilemma. Optionally, we could try to help the ansible-runner or python-daemon maintainers with new implementations of the problem dependencies as a way out. Whatever path we take, we're long overdue. The reasons we moved off lockfile ages ago are still there, and the risk to us has only continued to increase in the meantime. I'm open to suggestions, but we really ought to make sure we have it out of our constraints list by the Zed release. [0] https://discuss.python.org/t/17219 [1] https://lists.openstack.org/pipermail/openstack-dev/2014-June/038387.html [2] https://review.openstack.org/151224 [3] https://github.com/ansible/ansible-runner/issues/379 [4] https://pagure.io/python-daemon/issue/42 -- Jeremy Stanley
Hello there, Tripleo can't really drop the dependency on ansible-runner, since it's the official and supported way to run ansible from within python. We could of course replace it by subprocess.Popen and the whole family, but we'll lose all the facilities, especially the ones involved in the Ansible Execution Environment, which is kind of the "future" for running ansible (for the better and the worst). Since ansible-runner has an open issue for that (you even linked it), and at least one open pull-request to correct this issue, I'm not really sure we're needing to do anything but follow that open issue and ensure we get the right version of the package once they merge any of the proposal to remove it. Would that be OK? Though, of course, it won't allow to set a deadline on a specific date... Cheers, C. On 7/9/22 15:26, Jeremy Stanley wrote:
It became apparent in a Python community discussion[0] yesterday that lockfile has been designated as a "critical project" by PyPI, even though it was effectively abandoned in 2010. Because our release automation account is listed as one of its maintainers, I looked into whether we still need it for anything...
For those who have been around here a while, you may recall that the OpenStack project temporarily assumed maintenance[1] of lockfile in late 2014 and uploaded a few new releases for it, as a stop-gap until we could replace our uses with oslo.concurrency. That work was completed[2] early in the Liberty development cycle.
Unfortunately for us, that's not the end of the story. I looked yesterday expecting to see that we've not needed lockfile for 7 years, and was disappointed to discover it's still in our constraints list. Why? After much wailing and gnashing of teeth and manually installing multiple bisections of the requirements list, I narrowed it down to one dependency: ansible-runner.
Apparently, ansible-runner currently depends[3] on python-daemon, which still has a dependency on lockfile[4]. Our uses of ansible-runner seem to be pretty much limited to TripleO repositories (hence tagging them in the subject), so it's possible they could find an alternative to it and solve this dilemma. Optionally, we could try to help the ansible-runner or python-daemon maintainers with new implementations of the problem dependencies as a way out.
Whatever path we take, we're long overdue. The reasons we moved off lockfile ages ago are still there, and the risk to us has only continued to increase in the meantime. I'm open to suggestions, but we really ought to make sure we have it out of our constraints list by the Zed release.
[0] https://discuss.python.org/t/17219 [1] https://lists.openstack.org/pipermail/openstack-dev/2014-June/038387.html [2] https://review.openstack.org/151224 [3] https://github.com/ansible/ansible-runner/issues/379 [4] https://pagure.io/python-daemon/issue/42
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
We could, and maybe should, ask people working on ansible to give that issue/PR more attention. I see the PR is untriaged for some time now... On Mon, Jul 11, 2022 at 9:50 AM Cédric Jeanneret <cjeanner@redhat.com> wrote:
Hello there,
Tripleo can't really drop the dependency on ansible-runner, since it's the official and supported way to run ansible from within python. We could of course replace it by subprocess.Popen and the whole family, but we'll lose all the facilities, especially the ones involved in the Ansible Execution Environment, which is kind of the "future" for running ansible (for the better and the worst).
Since ansible-runner has an open issue for that (you even linked it), and at least one open pull-request to correct this issue, I'm not really sure we're needing to do anything but follow that open issue and ensure we get the right version of the package once they merge any of the proposal to remove it.
Would that be OK? Though, of course, it won't allow to set a deadline on a specific date...
Cheers,
C.
On 7/9/22 15:26, Jeremy Stanley wrote:
It became apparent in a Python community discussion[0] yesterday that lockfile has been designated as a "critical project" by PyPI, even though it was effectively abandoned in 2010. Because our release automation account is listed as one of its maintainers, I looked into whether we still need it for anything...
For those who have been around here a while, you may recall that the OpenStack project temporarily assumed maintenance[1] of lockfile in late 2014 and uploaded a few new releases for it, as a stop-gap until we could replace our uses with oslo.concurrency. That work was completed[2] early in the Liberty development cycle.
Unfortunately for us, that's not the end of the story. I looked yesterday expecting to see that we've not needed lockfile for 7 years, and was disappointed to discover it's still in our constraints list. Why? After much wailing and gnashing of teeth and manually installing multiple bisections of the requirements list, I narrowed it down to one dependency: ansible-runner.
Apparently, ansible-runner currently depends[3] on python-daemon, which still has a dependency on lockfile[4]. Our uses of ansible-runner seem to be pretty much limited to TripleO repositories (hence tagging them in the subject), so it's possible they could find an alternative to it and solve this dilemma. Optionally, we could try to help the ansible-runner or python-daemon maintainers with new implementations of the problem dependencies as a way out.
Whatever path we take, we're long overdue. The reasons we moved off lockfile ages ago are still there, and the risk to us has only continued to increase in the meantime. I'm open to suggestions, but we really ought to make sure we have it out of our constraints list by the Zed release.
[0] https://discuss.python.org/t/17219 [1] https://lists.openstack.org/pipermail/openstack-dev/2014-June/038387.html [2] https://review.openstack.org/151224 [3] https://github.com/ansible/ansible-runner/issues/379 [4] https://pagure.io/python-daemon/issue/42
-- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On 2022-07-09 13:26:36 +0000 (+0000), Jeremy Stanley wrote: [...]
Apparently, ansible-runner currently depends[3] on python-daemon, which still has a dependency on lockfile[4]. Our uses of ansible-runner seem to be pretty much limited to TripleO repositories (hence tagging them in the subject), so it's possible they could find an alternative to it and solve this dilemma. Optionally, we could try to help the ansible-runner or python-daemon maintainers with new implementations of the problem dependencies as a way out. [...]
In the meantime, how does everyone feel about us going ahead and removing the "openstackci" account from the maintainers list for lockfile on PyPI? We haven't depended on it directly since 2015, and it didn't come back into our indirect dependency set until 2018 (presumably that's when TripleO started using ansible-runner?). The odds that we'll need to fix anything in it in the future are pretty small at this point, and if we do we're better off putting that effort into helping the ansible-runner or python-daemon maintainers move off of it instead. -- Jeremy Stanley
On 7/16/22 03:52, Jeremy Stanley wrote:
On 2022-07-09 13:26:36 +0000 (+0000), Jeremy Stanley wrote: [...]
Apparently, ansible-runner currently depends[3] on python-daemon, which still has a dependency on lockfile[4]. Our uses of ansible-runner seem to be pretty much limited to TripleO repositories (hence tagging them in the subject), so it's possible they could find an alternative to it and solve this dilemma. Optionally, we could try to help the ansible-runner or python-daemon maintainers with new implementations of the problem dependencies as a way out. [...]
In the meantime, how does everyone feel about us going ahead and removing the "openstackci" account from the maintainers list for lockfile on PyPI? We haven't depended on it directly since 2015, and it didn't come back into our indirect dependency set until 2018 (presumably that's when TripleO started using ansible-runner?). The odds that we'll need to fix anything in it in the future are pretty small at this point, and if we do we're better off putting that effort into helping the ansible-runner or python-daemon maintainers move off of it instead.
That's a question for TripleO PTL - he's on holidays until next week (he was out also last week). It would be good to get his thoughts before doing anything, if it's possible. On my side, I don't really see any strong objection, but I may as well miss something important. Cheers, C. -- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/
On Mon, Jul 18, 2022 at 8:26 AM Cédric Jeanneret <cjeanner@redhat.com> wrote:
On 7/16/22 03:52, Jeremy Stanley wrote:
On 2022-07-09 13:26:36 +0000 (+0000), Jeremy Stanley wrote: [...]
Apparently, ansible-runner currently depends[3] on python-daemon, which still has a dependency on lockfile[4]. Our uses of ansible-runner seem to be pretty much limited to TripleO repositories (hence tagging them in the subject), so it's possible they could find an alternative to it and solve this dilemma. Optionally, we could try to help the ansible-runner or python-daemon maintainers with new implementations of the problem dependencies as a way out. [...]
In the meantime, how does everyone feel about us going ahead and removing the "openstackci" account from the maintainers list for lockfile on PyPI? We haven't depended on it directly since 2015, and it didn't come back into our indirect dependency set until 2018 (presumably that's when TripleO started using ansible-runner?). The odds that we'll need to fix anything in it in the future are pretty small at this point, and if we do we're better off putting that effort into helping the ansible-runner or python-daemon maintainers move off of it instead.
That's a question for TripleO PTL - he's on holidays until next week (he was out also last week). It would be good to get his thoughts before doing anything, if it's possible.
On my side, I don't really see any strong objection, but I may as well miss something important.
I don't have any objection to removing openstackci as a maintainer of lockfile on PypI. -- -- James Slagle --
On 2022-07-28 20:04:18 +0000 (+0000), Jeremy Stanley wrote:
On 2022-07-28 15:58:18 -0400 (-0400), James Slagle wrote: [...]
I don't have any objection to removing openstackci as a maintainer of lockfile on PypI.
Thanks for confirming! I'll get the ball rolling on that and let the original maintainer know.
And openstackci is no longer a maintainer of lockfile: https://discuss.python.org/t/17219/29 Thanks again, everyone! Now if we can just help the lockfile maintainer to wean everyone else off of it too... -- Jeremy Stanley
participants (4)
-
Cédric Jeanneret
-
James Slagle
-
Jeremy Stanley
-
Jiri Podivin