[all][refstack][neutron][kolla][ironic][heat][trove][senlin][barbican][manila] Fixing Zuul Config Errors
Hello, The OpenStack tenant in Zuul currently has 134 config errors. You can find these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of these errors appear related to project renames that have been requested of OpenDev or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate that a number of your jobs are probably not working). Project renames creating issues: * openstack/python-tempestconf -> osf/python-tempestconf -> openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack * x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw Project retirements creating issues: * openstack/neutron-lbaas * recordsansible/ara Projects whose configs have errors: * openinfra/python-tempestconf * openstack/heat * openstack/ironic * openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient Let us know if we can help decipher any errors, Clark
@ Kolla folks It seems to affect kolla-ansible in stable/rocky and stable/stein branches due to ansible_python_interpreter being set. We either need to backport https://review.opendev.org/c/openstack/kolla-ansible/+/798205 or drop affected CI config in those. They are EM so it might be fine going either way as CI might be broken by now anyhow. -yoctozepto On Tue, 16 Nov 2021 at 17:59, Clark Boylan <cboylan@sapwetik.org> wrote:
Hello,
The OpenStack tenant in Zuul currently has 134 config errors. You can find these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of these errors appear related to project renames that have been requested of OpenDev or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate that a number of your jobs are probably not working).
Project renames creating issues:
* openstack/python-tempestconf -> osf/python-tempestconf -> openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack * x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw
Project retirements creating issues:
* openstack/neutron-lbaas * recordsansible/ara
Projects whose configs have errors:
* openinfra/python-tempestconf * openstack/heat * openstack/ironic * openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient
Let us know if we can help decipher any errors, Clark
Hi, I just checked neutron related things there and it seems there are 2 major issues there: 1. move of the tap-as-a-service from x/ to openstack/ namespace (that affects networking-midonet) - I will propose patch for that today. 2. remove of the neutron-lbaas repo (that affects much more than only neutron repos - for that I will try to propose patches this week as well. On wtorek, 16 listopada 2021 17:58:44 CET Clark Boylan wrote:
Hello,
The OpenStack tenant in Zuul currently has 134 config errors. You can find these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of these errors appear related to project renames that have been requested of OpenDev or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate that a number of your jobs are probably not working).
Project renames creating issues:
* openstack/python-tempestconf -> osf/python-tempestconf -> openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack * x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw
Project retirements creating issues:
* openstack/neutron-lbaas * recordsansible/ara
Projects whose configs have errors:
* openinfra/python-tempestconf * openstack/heat * openstack/ironic * openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient
Let us know if we can help decipher any errors, Clark
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hi, On środa, 17 listopada 2021 08:26:14 CET Slawek Kaplonski wrote:
Hi,
I just checked neutron related things there and it seems there are 2 major issues there: 1. move of the tap-as-a-service from x/ to openstack/ namespace (that affects networking-midonet) - I will propose patch for that today. 2. remove of the neutron-lbaas repo (that affects much more than only neutron repos - for that I will try to propose patches this week as well.
There are also some missing job definitions in some of the neutron related repos and also issues with missing openstack/networking-l2gw project. I will take a look into all those issues in next days.
On wtorek, 16 listopada 2021 17:58:44 CET Clark Boylan wrote:
Hello,
The OpenStack tenant in Zuul currently has 134 config errors. You can find these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of these errors appear related to project renames that have been requested of
or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate
OpenDev that
a number of your jobs are probably not working).
Project renames creating issues: * openstack/python-tempestconf -> osf/python-tempestconf ->
openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack
* x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw
Project retirements creating issues: * openstack/neutron-lbaas * recordsansible/ara
Projects whose configs have errors: * openinfra/python-tempestconf * openstack/heat * openstack/ironic * openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient
Let us know if we can help decipher any errors, Clark
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hi Clark, Why is the retirement of openstack/neutron-lbaas being a problem? The repo is there and accessible under the same URL, it has (potentially working) stable/pike and stable/queens branches, and was not retired at the time of Pike or Queens, so IMO it is a valid request for testing configuration in the same branches of other projects, openstack/heat in this case. Maybe we should leave some minimal zuul configs in retired projects for zuul to find them? Cheers, On Wed, Nov 17, 2021 at 9:45 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
On środa, 17 listopada 2021 08:26:14 CET Slawek Kaplonski wrote:
Hi,
I just checked neutron related things there and it seems there are 2 major issues there: 1. move of the tap-as-a-service from x/ to openstack/ namespace (that affects networking-midonet) - I will propose patch for that today. 2. remove of the neutron-lbaas repo (that affects much more than only neutron repos - for that I will try to propose patches this week as well.
There are also some missing job definitions in some of the neutron related repos and also issues with missing openstack/networking-l2gw project. I will take a look into all those issues in next days.
On wtorek, 16 listopada 2021 17:58:44 CET Clark Boylan wrote:
Hello,
The OpenStack tenant in Zuul currently has 134 config errors. You can
these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of
errors appear related to project renames that have been requested of OpenDev or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate
find these that
a number of your jobs are probably not working).
Project renames creating issues: * openstack/python-tempestconf -> osf/python-tempestconf ->
openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack
* x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw
Project retirements creating issues: * openstack/neutron-lbaas * recordsansible/ara
Projects whose configs have errors: * openinfra/python-tempestconf * openstack/heat * openstack/ironic * openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient
Let us know if we can help decipher any errors, Clark
-- Slawek Kaplonski Principal Software Engineer Red Hat
-- Dr. Pavlo Shchelokovskyy Principal Software Engineer Mirantis Inc www.mirantis.com
On Thu, Nov 18, 2021, at 3:11 AM, Pavlo Shchelokovskyy wrote:
Hi Clark,
Why is the retirement of openstack/neutron-lbaas being a problem?
The repo is there and accessible under the same URL, it has (potentially working) stable/pike and stable/queens branches, and was not retired at the time of Pike or Queens, so IMO it is a valid request for testing configuration in the same branches of other projects, openstack/heat in this case.
Maybe we should leave some minimal zuul configs in retired projects for zuul to find them?
The reason for this is one of the steps for project retirement is to remove the repo from zuul [0]. If the upstream for the project has retired the project I think it is reasonable to remove it from our systems like Zuul. The project isn't retired on a per branch basis. I do not think the Zuul operators should be responsible for keeping retired projects alive in the CI system if their maintainers have stopped maintaining them. Instead we should remove them from the CI system. [0] https://opendev.org/openstack/project-config/commit/3832c8fafc4d4e03c306c21f...
---- On Thu, 18 Nov 2021 08:40:38 -0600 Clark Boylan <cboylan@sapwetik.org> wrote ----
On Thu, Nov 18, 2021, at 3:11 AM, Pavlo Shchelokovskyy wrote:
Hi Clark,
Why is the retirement of openstack/neutron-lbaas being a problem?
The repo is there and accessible under the same URL, it has (potentially working) stable/pike and stable/queens branches, and was not retired at the time of Pike or Queens, so IMO it is a valid request for testing configuration in the same branches of other projects, openstack/heat in this case.
Maybe we should leave some minimal zuul configs in retired projects for zuul to find them?
The reason for this is one of the steps for project retirement is to remove the repo from zuul [0]. If the upstream for the project has retired the project I think it is reasonable to remove it from our systems like Zuul. The project isn't retired on a per branch basis.
I do not think the Zuul operators should be responsible for keeping retired projects alive in the CI system if their maintainers have stopped maintaining them. Instead we should remove them from the CI system.
I agree, in case of deprecation we take care of stable branch and in retirement case, it means it is gone completely so usage of retired repo has to be completely cleanup from everywhere (master or stable). -gmann
[0] https://opendev.org/openstack/project-config/commit/3832c8fafc4d4e03c306c21f...
---- On Fri, 19 Nov 2021 14:59:45 -0600 Ghanshyam Mann <gmann@ghanshyammann.com> wrote ----
---- On Thu, 18 Nov 2021 08:40:38 -0600 Clark Boylan <cboylan@sapwetik.org> wrote ----
On Thu, Nov 18, 2021, at 3:11 AM, Pavlo Shchelokovskyy wrote:
Hi Clark,
Why is the retirement of openstack/neutron-lbaas being a problem?
The repo is there and accessible under the same URL, it has (potentially working) stable/pike and stable/queens branches, and was not retired at the time of Pike or Queens, so IMO it is a valid request for testing configuration in the same branches of other projects, openstack/heat in this case.
Maybe we should leave some minimal zuul configs in retired projects for zuul to find them?
The reason for this is one of the steps for project retirement is to remove the repo from zuul [0]. If the upstream for the project has retired the project I think it is reasonable to remove it from our systems like Zuul. The project isn't retired on a per branch basis.
I do not think the Zuul operators should be responsible for keeping retired projects alive in the CI system if their maintainers have stopped maintaining them. Instead we should remove them from the CI system.
I agree, in case of deprecation we take care of stable branch and in retirement case, it means it is gone completely so usage of retired repo has to be completely cleanup from everywhere (master or stable).
I am adding this to the TC meeting agenda so that we do not forget it. Also, created the below etherpad to track the progress, please add the progress and patch links under your project: - https://etherpad.opendev.org/p/zuul-config-error-openstack -gmann
-gmann
[0] https://opendev.org/openstack/project-config/commit/3832c8fafc4d4e03c306c21f...
On Tue, Nov 16, 2021 at 6:05 PM Clark Boylan <cboylan@sapwetik.org> wrote:
Hello,
The OpenStack tenant in Zuul currently has 134 config errors. You can find these errors at https://zuul.opendev.org/t/openstack/config-errors or by clicking the blue bell icon in the top right of https://zuul.opendev.org/t/openstack/status. The vast majority of these errors appear related to project renames that have been requested of OpenDev or project retirements. Can you please look into fixing these as they can be an attractive nuisance when debugging Zuul problems (they also indicate that a number of your jobs are probably not working).
Project renames creating issues:
* openstack/python-tempestconf -> osf/python-tempestconf -> openinfra/python-tempestconf * openstack/refstack -> osf/refstack -> openinfra/refstack * x/tap-as-a-service -> openstack/tap-as-a-service * openstack/networking-l2gw -> x/networking-l2gw
Project retirements creating issues:
* openstack/neutron-lbaas * recordsansible/ara
Projects whose configs have errors:
* openinfra/python-tempestconf * openstack/heat * openstack/ironic
Ironic failures are missing jobs on Queens and Rocky. I'd avoid touching these branches unless our problems are breaking someone. Dmitry
* openstack/kolla-ansible * openstack/kuryr-kubernetes * openstack/murano-apps * openstack/networking-midonet * openstack/networking-odl * openstack/neutron * openstack/neutron-fwaas * openstack/python-troveclient * openstack/senlin * openstack/tap-as-a-service * openstack/zaqar * x/vmware-nsx * openinfra/openstackid * openstack/barbican * openstack/cookbook-openstack-application-catalog * openstack/heat-dashboard * openstack/manila-ui * openstack/python-manilaclient
Let us know if we can help decipher any errors, Clark
-- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
participants (6)
-
Clark Boylan
-
Dmitry Tantsur
-
Ghanshyam Mann
-
Pavlo Shchelokovskyy
-
Radosław Piliszek
-
Slawek Kaplonski