Upcoming pip changes - stop installing test-requirements in devstack
Heya, Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us. The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent. I’ve pushed up this: https://review.opendev.org/#/c/715469/ Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now. It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer. Monty
Hiya, it's now going in. Dev Folks: if it breaks jobs and you need assistance, please respond to this thread or ping on #openstack-qa -yoctozepto pt., 27 mar 2020 o 20:33 Monty Taylor <mordred@inaugust.com> napisał(a):
Heya,
Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us.
The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent.
I’ve pushed up this:
https://review.opendev.org/#/c/715469/
Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now.
It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer.
Monty
On Fri, 2020-03-27 at 14:24 -0500, Monty Taylor wrote:
Heya,
Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us.
The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent.
I’ve pushed up this:
https://review.opendev.org/#/c/715469/
Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now.
i brought up the topic of stoping installing test-requirement in the past as it has caused issue before so im happy to see this change going forward :)
It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer.
Monty
---- On Sat, 28 Mar 2020 13:57:38 -0500 Sean Mooney <smooney@redhat.com> wrote ----
On Fri, 2020-03-27 at 14:24 -0500, Monty Taylor wrote:
Heya,
Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us.
The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent.
I’ve pushed up this:
https://review.opendev.org/#/c/715469/
Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now.
i brought up the topic of stoping installing test-requirement in the past as it has caused issue before so im happy to see this change going forward :)
It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer.
Currently, neutron gate is red for glance_store missing requirement of swift/cinderclients which used to get installed by test-requirements installation. Fixes we are trying: - https://review.opendev.org/#/c/715835/2 - https://review.opendev.org/#/c/715722/1 Swift functional jobs are also failing on py2 things due to stestr drop py2 support so capping of stestr for py2 is up - https://review.opendev.org/#/c/715942/ let us know on this thread or on #openstack-qa if any other project failing. We are trying to merge those asap and if that fails further or take time we will go with temporary revert of test-requirement drop patch -gmann
Monty
---- On Mon, 30 Mar 2020 09:03:10 -0500 Ghanshyam Mann <gmann@ghanshyammann.com> wrote ----
---- On Sat, 28 Mar 2020 13:57:38 -0500 Sean Mooney <smooney@redhat.com> wrote ----
On Fri, 2020-03-27 at 14:24 -0500, Monty Taylor wrote:
Heya,
Recently we had to revert a change to devstack that ran pip check. This is a recommendation from the pip team to verify that upcoming addition of a depsolver to pip isn’t going to break us. Well - it is going to break us.
The reason it’s going to break us is that we don’t apply constraints to linters (so that projects can deal with those at their own rate) BUT - we install every project’s test-requirements.txt (containing the linters) when we run devstack. That means we uninstall and reinstall flake8 at different versions over and over again - and the final state is not one that is completely consistent.
I’ve pushed up this:
https://review.opendev.org/#/c/715469/
Which is a patch to stop installing test-requirements in devstack. Realistically we should not need to install this in devstack. Devstack is installing the software which does not need test-requirements - and then we test it with tempest which has its own set of requirements. As a side benefit, devstack is quicker and none of the non-voting jobs on devstack fail now.
i brought up the topic of stoping installing test-requirement in the past as it has caused issue before so im happy to see this change going forward :)
It’s possible this change could impact people using devstack as a base for single-project functional tests - so it’s worth being careful with. But I think we should get it done and then re-add the pip check as a verification that we’re not going to get broken this summer.
Currently, neutron gate is red for glance_store missing requirement of swift/cinderclients which used to get installed by test-requirements installation. Fixes we are trying: - https://review.opendev.org/#/c/715835/2 - https://review.opendev.org/#/c/715722/1
Swift functional jobs are also failing on py2 things due to stestr drop py2 support so capping of stestr for py2 is up - https://review.opendev.org/#/c/715942/
You might have noticed, both fixes are merged now, we are good to recheck. -gmann
let us know on this thread or on #openstack-qa if any other project failing.
We are trying to merge those asap and if that fails further or take time we will go with temporary revert of test-requirement drop patch
-gmann
Monty
participants (4)
-
Ghanshyam Mann
-
Monty Taylor
-
Radosław Piliszek
-
Sean Mooney