[dev] [all] [ptl] Migrating legacy jobs to Bionic (Ubuntu LTS 18.04)
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs. 1. Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/ 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
Ghanshyam Mann gmann@ghanshyammann.com writes:
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
Thanks for driving this, Ghanshyam, and helping to ensure we're all testing on a consistent and current platform.
Hi,
I recently did such DNM patches for Neutron and Neutron stadium projects. List of patches available on [1]. There is couple of bugs related to this switch reported:
* Neutron: https://bugs.launchpad.net/neutron/+bug/1818632 * Networking-ovn: https://bugs.launchpad.net/networking-ovn/+bug/1818626 * Networking-bagpipe: https://bugs.launchpad.net/networking-bagpipe/+bug/1818624 * Neutron-dynamic-routing: https://bugs.launchpad.net/neutron/+bug/1818628
For other projects it looks for me that it’s good (or at least it’s not getting to be worst with Bionic :)) but if You are more familiar with some of stadium projects, please take a look at those patches and jobs’ results. Maybe You will find something what I missed.
[1] https://review.openstack.org/#/q/topic:legacy-job-bionic+status:open+owner:%...
Wiadomość napisana przez Ghanshyam Mann gmann@ghanshyammann.com w dniu 25.02.2019, o godz. 16:05:
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
— Slawek Kaplonski Senior software engineer Red Hat
I have run jobs for both Octavia and neutron-lbaas and the legacy jobs all passed. I have updated the ehterpad.
Thanks for the heads up. Michael
On Tue, Mar 5, 2019 at 2:43 AM Slawomir Kaplonski skaplons@redhat.com wrote:
Hi,
I recently did such DNM patches for Neutron and Neutron stadium projects. List of patches available on [1]. There is couple of bugs related to this switch reported:
- Neutron: https://bugs.launchpad.net/neutron/+bug/1818632
- Networking-ovn: https://bugs.launchpad.net/networking-ovn/+bug/1818626
- Networking-bagpipe: https://bugs.launchpad.net/networking-bagpipe/+bug/1818624
- Neutron-dynamic-routing: https://bugs.launchpad.net/neutron/+bug/1818628
For other projects it looks for me that it’s good (or at least it’s not getting to be worst with Bionic :)) but if You are more familiar with some of stadium projects, please take a look at those patches and jobs’ results. Maybe You will find something what I missed.
[1] https://review.openstack.org/#/q/topic:legacy-job-bionic+status:open+owner:%...
Wiadomość napisana przez Ghanshyam Mann gmann@ghanshyammann.com w dniu 25.02.2019, o godz. 16:05:
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
— Slawek Kaplonski Senior software engineer Red Hat
---- On Wed, 06 Mar 2019 10:54:56 -0600 Michael Johnson johnsomor@gmail.com wrote ----
I have run jobs for both Octavia and neutron-lbaas and the legacy jobs all passed. I have updated the ehterpad.
Thanks for the heads up. Michael
On Tue, Mar 5, 2019 at 2:43 AM Slawomir Kaplonski skaplons@redhat.com wrote:
Hi,
I recently did such DNM patches for Neutron and Neutron stadium projects. List of patches available on [1]. There is couple of bugs related to this switch reported:
- Neutron: https://bugs.launchpad.net/neutron/+bug/1818632
- Networking-ovn: https://bugs.launchpad.net/networking-ovn/+bug/1818626
- Networking-bagpipe: https://bugs.launchpad.net/networking-bagpipe/+bug/1818624
- Neutron-dynamic-routing: https://bugs.launchpad.net/neutron/+bug/1818628
For other projects it looks for me that it’s good (or at least it’s not getting to be worst with Bionic :)) but if You are more familiar with some of stadium projects, please take a look at those patches and jobs’ results. Maybe You will find something what I missed.
[1] https://review.openstack.org/#/q/topic:legacy-job-bionic+status:open+owner:%...
Wiadomość napisana przez Ghanshyam Mann gmann@ghanshyammann.com w dniu 25.02.2019, o godz. 16:05:
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
— Slawek Kaplonski Senior software engineer Red Hat
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic: -------------------------------------------------- Step 1. Push the testing DNM patch on your project owned repos with: - Depends-On: https://review.openstack.org/#/c/641886/ -Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later. --------------------------------------------------
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
Hello Everyone.
Updates on testing of legacy jobs on Bionic:
I have pushed the patches to projects repo which has legacy jobs - https://review.openstack.org/#/q/topic:legacy-job-bionic+(status:open+OR+sta...)
Also updated the link in etherpad. - https://etherpad.openstack.org/p/legacy-job-bionic
gate results of those patches should be ready by night. Please check your projects patch and mark OK in etherpad or fix the bugs if any before March 10th.
*Testing Started or no legacy jobs found until now*: 39 QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun Qinling : OK. no legacy jobs Searchlight : OK. no legacy jobs Senlin : OK. no legacy jobs Solum : OK. no legacy jobs Storlets : OK. no legacy jobs Winstackers: OK. no legacy jobs Adjutant- OK. no legacy jobs
*Testing not started*: 24 Chef Openstack (Chef cookbooks for deployment) Cloudkitty (Rating service) Documentation Freezer (Backup, Restore, and Disaster Recovery service) Horizon (Dashboard) I18n Infrastructure Kolla Loci Openstack Charms (Juju Charms for deployment of OpenStack) Openstack-Helm (Helm charts for OpenStack services) Openstackansible (Ansible playbooks and roles for deployment) Openstackclient (Command-line client) Openstacksdk (Multi-cloud Python SDK for End Users) Packaging-Rpm Powervmstackers Puppet Openstack (Puppet modules for deployment) Rally (Benchmark service) Release Management Requirements Swift (Object Storage service) Tacker (NFV Orchestration service) Tricircle (Networking automation across Neutron service) Tripleo (Deployment service)
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hello Everyone.
Updates on testing of legacy jobs on Bionic:
I have pushed the patches to projects repo which has legacy jobs
Also updated the link in etherpad. - https://etherpad.openstack.org/p/legacy-job-bionic
gate results of those patches should be ready by night. Please check your projects patch and mark OK in etherpad or fix the bugs if any before March 10th.
*Testing Started or no legacy jobs found until now*: 39 QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun Qinling : OK. no legacy jobs Searchlight : OK. no legacy jobs Senlin : OK. no legacy jobs Solum : OK. no legacy jobs Storlets : OK. no legacy jobs Winstackers: OK. no legacy jobs Adjutant- OK. no legacy jobs
*Testing not started*: 24 Chef Openstack (Chef cookbooks for deployment) Cloudkitty (Rating service) Documentation Freezer (Backup, Restore, and Disaster Recovery service) Horizon (Dashboard) I18n Infrastructure Kolla Loci Openstack Charms (Juju Charms for deployment of OpenStack) Openstack-Helm (Helm charts for OpenStack services) Openstackansible (Ansible playbooks and roles for deployment) Openstackclient (Command-line client) Openstacksdk (Multi-cloud Python SDK for End Users) Packaging-Rpm Powervmstackers Puppet Openstack (Puppet modules for deployment) Rally (Benchmark service) Release Management Requirements Swift (Object Storage service) Tacker (NFV Orchestration service) Tricircle (Networking automation across Neutron service) Tripleo (Deployment service)
-gmann
Updates:
I pushed few of the remaining projects testing patch today, below is the latest status. Request you to check your project testing patch and take appropriate action. March 13th is deadline to merge the base legacy jobs migration patch.
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
- https://etherpad.openstack.org/p/legacy-job-bionic
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
I pushed few of the remaining projects testing patch today, below is the latest status. Request you to check your project testing patch and take appropriate action. March 13th is deadline to merge the base legacy jobs migration patch.
[...]
Updates:
I checked the project which are passing their gate on Bionic and which are failed (have not checked the n-v or experimental jobs) - https://review.openstack.org/#/q/topic:legacy-job-bionic+(status:open+OR+sta...)
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/ freezer - not able to connect elasticsearch service - https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
---- On Wed, 13 Mar 2019 00:21:46 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
[...]
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/ freezer - not able to connect elasticsearch service
- https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
I am planning to merge the base patches in another 2.5 hours (21:00 UTC today). Try to check your gate failure (mentioned above) before that, otherwise, as last option I will push the patch to make failed jobs as n-v.
-gmann
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
On 13/03/2019 18:27, Ghanshyam Mann wrote:
---- On Wed, 13 Mar 2019 00:21:46 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
[...]
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/ freezer - not able to connect elasticsearch service
- https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
I am planning to merge the base patches in another 2.5 hours (21:00 UTC today). Try to check your gate failure (mentioned above) before that, otherwise, as last option I will push the patch to make failed jobs as n-v.
-gmann
We (designate) think we have the fix for ours [1] and I have a patch up to test it as well [2].
Don't worry about setting our jobs to n-v - it is a blocker for us, so nothing should merge until it is fixed.
1 - https://review.openstack.org/643119/ 2 - https://review.openstack.org/643127
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > Hi Everyone, > > During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1]. > > But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. > As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html > This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2]. > > I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the > Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic > > The approach is the same as we did for previous migration for devstack jobs. > 1. Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/ > 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status > of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 > 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own. > > Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign > someone from their team if no volunteer. > > Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. > let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release. > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... > https://etherpad.openstack.org/p/devstack-bionic > > [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html > > > -gmann > > >
---- On Wed, 13 Mar 2019 13:34:29 -0500 Graham Hayes gr@ham.ie wrote ----
On 13/03/2019 18:27, Ghanshyam Mann wrote:
---- On Wed, 13 Mar 2019 00:21:46 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
[...]
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/ freezer - not able to connect elasticsearch service
- https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
I am planning to merge the base patches in another 2.5 hours (21:00 UTC today). Try to check your gate failure (mentioned above) before that, otherwise, as last option I will push the patch to make failed jobs as n-v.
-gmann
We (designate) think we have the fix for ours [1] and I have a patch up to test it as well [2].
Don't worry about setting our jobs to n-v - it is a blocker for us, so nothing should merge until it is fixed.
1 - https://review.openstack.org/643119/ 2 - https://review.openstack.org/643127
ACK, thanks.
Cinder also testing the fixes so I leave Cinder also from making job as n-v.
- https://review.openstack.org/#/c/643130/ - https://review.openstack.org/#/c/641926/
-gmann
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > Thanks all who volunteer for their project and testing the jobs on bionic. > > Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. > I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification. > > Thanks. > > -gmann > > ---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > Hi Everyone, > > > > During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1]. > > > > But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. > > As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html > > This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2]. > > > > I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the > > Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic > > > > The approach is the same as we did for previous migration for devstack jobs. > > 1. Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/ > > 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status > > of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 > > 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own. > > > > Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign > > someone from their team if no volunteer. > > > > Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. > > let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release. > > > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... > > https://etherpad.openstack.org/p/devstack-bionic > > > > [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html > > > > > > -gmann > > > > > > > >
---- On Wed, 13 Mar 2019 13:45:19 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Wed, 13 Mar 2019 13:34:29 -0500 Graham Hayes gr@ham.ie wrote ----
On 13/03/2019 18:27, Ghanshyam Mann wrote:
---- On Wed, 13 Mar 2019 00:21:46 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
[...]
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/ freezer - not able to connect elasticsearch service
- https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
I am planning to merge the base patches in another 2.5 hours (21:00 UTC today). Try to check your gate failure (mentioned above) before that, otherwise, as last option I will push the patch to make failed jobs as n-v.
-gmann
We (designate) think we have the fix for ours [1] and I have a patch up to test it as well [2].
Don't worry about setting our jobs to n-v - it is a blocker for us, so nothing should merge until it is fixed.
1 - https://review.openstack.org/643119/ 2 - https://review.openstack.org/643127
ACK, thanks.
Cinder also testing the fixes so I leave Cinder also from making job as n-v.
Base patches are merged now - https://review.openstack.org/#/c/639096/ - https://review.openstack.org/#/c/641886/2
If you find any of your gate job failing and need help, ping us on #openstack-qa or #openstack-infra or #openstack-tc channel.
Projects need to merge these patches soon to avoid gate block Nova - https://review.openstack.org/#/c/643129/. - Already approved. in gate ti merge. Cinder - https://review.openstack.org/#/c/643130/ - testing in progress. Designate - https://review.openstack.org/643119/ already approved. in gate to merge. Trove - no effect until this merge which need more work - https://review.openstack.org/#/c/642329/ Zaqar - https://review.openstack.org/#/c/642349/1 testing it now. networking-midonet - https://review.openstack.org/#/c/643022/3 freezer - need to fix.
-gmann
-gmann
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > Updates on planning and dadlines: Legacy jobs to bionic. > > I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. > Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop > the py35 testing. > > NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. > ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic. > > DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix. > > How to test your project's legacy jobs : > I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. > Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic > > For quick reference, I am writting the steps to test your legacy jobs on bionic: > -------------------------------------------------- > Step 1. Push the testing DNM patch on your project owned repos with: > - Depends-On: https://review.openstack.org/#/c/641886/ > -Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) > Example: https://review.openstack.org/#/c/639017 > > Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ > Example - https://review.openstack.org/#/c/639361/ > > Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later. > -------------------------------------------------- > > Feel free to ping me for any query or help. > > [1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... > [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm... > > -gmann > > > ---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > Thanks all who volunteer for their project and testing the jobs on bionic. > > > > Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. > > I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification. > > > > Thanks. > > > > -gmann > > > > ---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > > Hi Everyone, > > > > > > During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1]. > > > > > > But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. > > > As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html > > > This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2]. > > > > > > I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the > > > Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic > > > > > > The approach is the same as we did for previous migration for devstack jobs. > > > 1. Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/ > > > 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status > > > of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 > > > 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own. > > > > > > Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign > > > someone from their team if no volunteer. > > > > > > Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. > > > let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release. > > > > > > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... > > > https://etherpad.openstack.org/p/devstack-bionic > > > > > > [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html > > > > > > > > > -gmann > > > > > > > > > > > > > > >
---- On Wed, 13 Mar 2019 16:45:04 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Base patches are merged now Projects need to merge these patches soon to avoid gate block Nova
- https://review.openstack.org/#/c/643129/. - Already approved. in gate ti merge.
Cinder
- https://review.openstack.org/#/c/643130/ - testing in progress.
Designate
- https://review.openstack.org/643119/ already approved. in gate to merge.
Trove
- no effect until this merge which need more work - https://review.openstack.org/#/c/642329/
Zaqar - https://review.openstack.org/#/c/642349/1 testing it now. networking-midonet - https://review.openstack.org/#/c/643022/3 freezer - need to fix.
[...]
Updates: Nova- workaround merged. Gate is up Cinder - devstack fix is in gate to merge. devstack unit test broken now on bionic. Designate - Fix is up in gate. Trove - no gate failure due to base patches, Bionic migration in progress Zaqar- No failure, Gate is up networking-midonet - workaround is up, Gate failing for other reason. Freezer - workaround merged, Gate is up
I have abandoned the DNM testing patch.
Below are the patches up for review which are needed to migrate the derived legacy jobs to Bionic: - https://review.openstack.org/#/q/topic:legacy-job-bionic+status:open
The corresponding projects need to merge them to complete this migration.
-gmann
-gmann
-gmann
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
> > ---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > Updates on planning and dadlines: Legacy jobs to bionic. > > > > I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. > > Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop > > the py35 testing. > > > > NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. > > ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic. > > > > DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix. > > > > How to test your project's legacy jobs : > > I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. > > Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic > > > > For quick reference, I am writting the steps to test your legacy jobs on bionic: > > -------------------------------------------------- > > Step 1. Push the testing DNM patch on your project owned repos with: > > - Depends-On: https://review.openstack.org/#/c/641886/ > > -Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) > > Example: https://review.openstack.org/#/c/639017 > > > > Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ > > Example - https://review.openstack.org/#/c/639361/ > > > > Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later. > > -------------------------------------------------- > > > > Feel free to ping me for any query or help. > > > > [1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... > > [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm... > > > > -gmann > > > > > > ---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > > Thanks all who volunteer for their project and testing the jobs on bionic. > > > > > > Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. > > > I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification. > > > > > > Thanks. > > > > > > -gmann > > > > > > ---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > > > Hi Everyone, > > > > > > > > During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1]. > > > > > > > > But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. > > > > As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html > > > > This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2]. > > > > > > > > I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the > > > > Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic > > > > > > > > The approach is the same as we did for previous migration for devstack jobs. > > > > 1. Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/ > > > > 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status > > > > of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 > > > > 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own. > > > > > > > > Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign > > > > someone from their team if no volunteer. > > > > > > > > Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. > > > > let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release. > > > > > > > > > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... > > > > https://etherpad.openstack.org/p/devstack-bionic > > > > > > > > [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html > > > > > > > > > > > > -gmann > > > > > > > > > > > > > > > > > > > > > > > > > >
On 3/13/2019 12:21 AM, Ghanshyam Mann wrote:
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable -https://review.openstack.org/#/c/639017/
This is being worked around for two separate issues:
1. ceph setup in the nova-live-migration and nova-grenade-live-migration job isn't working with bionic: https://review.openstack.org/#/c/643122/
2. tls-proxy is being disabled in the nova-next job as a workaround: https://review.openstack.org/#/c/643129/
I'm not sure what's going on with #1 since it's using the devstack-plugin-ceph repo functions to install ceph but it's not very easy to tell what's going wrong from the logs, we probably need a debug patch which turns on xtrace.
As for the tls-proxy one, it looks like Mel is investigating it a bit and could probably also use some of Stephen's help.
On Wed, 13 Mar 2019 13:32:01 -0500, Matt Riedemann mriedemos@gmail.com wrote:
On 3/13/2019 12:21 AM, Ghanshyam Mann wrote:
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable -https://review.openstack.org/#/c/639017/
This is being worked around for two separate issues:
- ceph setup in the nova-live-migration and nova-grenade-live-migration
job isn't working with bionic: https://review.openstack.org/#/c/643122/
- tls-proxy is being disabled in the nova-next job as a workaround:
https://review.openstack.org/#/c/643129/
I'm not sure what's going on with #1 since it's using the devstack-plugin-ceph repo functions to install ceph but it's not very easy to tell what's going wrong from the logs, we probably need a debug patch which turns on xtrace.
From what it says in the commit message on the skip patch, the plugin wants to install Ceph Hammer (fairly old version) and that version is not available on bionic. I'll look into whether we can move up to a newer version (or something) to fix this issue.
As for the tls-proxy one, it looks like Mel is investigating it a bit and could probably also use some of Stephen's help.
Yeah, I'm digging into it. I first tried an attempt to use 2048 bit certs instead of 1024 bit, based on this bugzilla I found:
https://bugzilla.redhat.com/show_bug.cgi?id=1651882#c6 - "The certificates which are generated by the script are too weak for openssl default's configuration, and thus they get rejected."
but that didn't work. Next, I'm going to see if it's a permission issue with the ownership of the certs directory. I remember when I originally tweaked Dan Berrange's patch to run console proxies with TLS in CI, I had to change the user:group from qemu:qemu to libvirt-qemu:libvirt-qemu [1] before it worked. So, I'm wondering if there's possibly new change in the qemu user, or something like that. I'll keep investigating.
-melanie
[1] https://review.openstack.org/#/c/396186/6..7/lib/nova_plugins/functions-libv...
On Wed, 13 Mar 2019 12:28:23 -0700, Melanie Witt melwittt@gmail.com wrote:
On Wed, 13 Mar 2019 13:32:01 -0500, Matt Riedemann mriedemos@gmail.com wrote:
On 3/13/2019 12:21 AM, Ghanshyam Mann wrote:
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable -https://review.openstack.org/#/c/639017/
This is being worked around for two separate issues:
- ceph setup in the nova-live-migration and nova-grenade-live-migration
job isn't working with bionic: https://review.openstack.org/#/c/643122/
- tls-proxy is being disabled in the nova-next job as a workaround:
https://review.openstack.org/#/c/643129/
I'm not sure what's going on with #1 since it's using the devstack-plugin-ceph repo functions to install ceph but it's not very easy to tell what's going wrong from the logs, we probably need a debug patch which turns on xtrace.
From what it says in the commit message on the skip patch, the plugin wants to install Ceph Hammer (fairly old version) and that version is not available on bionic. I'll look into whether we can move up to a newer version (or something) to fix this issue.
FYI to thread readers, this got fixed by:
https://review.openstack.org/643150
As for the tls-proxy one, it looks like Mel is investigating it a bit and could probably also use some of Stephen's help.
Yeah, I'm digging into it. I first tried an attempt to use 2048 bit certs instead of 1024 bit, based on this bugzilla I found:
https://bugzilla.redhat.com/show_bug.cgi?id=1651882#c6 - "The certificates which are generated by the script are too weak for openssl default's configuration, and thus they get rejected."
but that didn't work. Next, I'm going to see if it's a permission issue with the ownership of the certs directory. I remember when I originally tweaked Dan Berrange's patch to run console proxies with TLS in CI, I had to change the user:group from qemu:qemu to libvirt-qemu:libvirt-qemu [1] before it worked. So, I'm wondering if there's possibly new change in the qemu user, or something like that. I'll keep investigating.
[1] https://review.openstack.org/#/c/396186/6..7/lib/nova_plugins/functions-libv...
I finally got the console testing with TLS issue worked out, patches are proposed here:
https://review.openstack.org/643045 https://review.openstack.org/645432
-melanie
On Wed, Mar 13, 2019 at 2:25 PM Ghanshyam Mann gmann@ghanshyammann.com wrote:
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
I pushed few of the remaining projects testing patch today, below is the latest status. Request you to check your project testing patch and take appropriate action. March 13th is deadline to merge the base legacy jobs migration patch.
[...]
Updates:
I checked the project which are passing their gate on Bionic and which are failed (have not checked the n-v or experimental jobs)
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/
is there an option to keep xenial jobs? midonet (the backend networking-midonet talks to) doesn't support bionic yet. https://midonet.atlassian.net/browse/MNA-1344
freezer - not able to connect elasticsearch service
- https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Hi Everyone,
During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1].
But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2].
I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic
The approach is the same as we did for previous migration for devstack jobs.
- Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/
- Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status of their gate on etherpad - Example: https://review.openstack.org/#/c/639017
- Project jobs not using base job as parent, need to migrate their legacy job to bionic by own.
Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign someone from their team if no volunteer.
Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... https://etherpad.openstack.org/p/devstack-bionic
[2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html
-gmann
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/
is there an option to keep xenial jobs? midonet (the backend networking-midonet talks to) doesn't support bionic yet. https://midonet.atlassian.net/browse/MNA-1344
You may need to add the following line to the impacted networking-midonet jobs:
nodeset: ubuntu-xenial
There is an example of this being done here:
https://review.openstack.org/#/c/643375/
This should probably be considered a short term workaround though. I'm curious how this is going to be for end users if the package is not available on newer distros.
---- On Thu, 14 Mar 2019 23:37:46 -0500 Takashi Yamamoto yamamoto@midokura.com wrote ----
On Wed, Mar 13, 2019 at 2:25 PM Ghanshyam Mann gmann@ghanshyammann.com wrote:
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
I pushed few of the remaining projects testing patch today, below is the latest status. Request you to check your project testing patch and take appropriate action. March 13th is deadline to merge the base legacy jobs migration patch.
[...]
Updates:
I checked the project which are passing their gate on Bionic and which are failed (have not checked the n-v or experimental jobs)
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/
is there an option to keep xenial jobs? midonet (the backend networking-midonet talks to) doesn't support bionic yet. https://midonet.atlassian.net/browse/MNA-1344
We really want to avoid that. Main idea if doing this migration at community level is that all the OpenStack projects which are integrated with each other at some level can move and be tested on latest Distro. So that they do not break each other.
In your case, neutron is integrated projects which is or going to be running/tested on Bionic and leaving networking-midonet on xenial can leads to integrated issues at some time. As I have proposed the patch [1], we move the job to Bionic and make it n-v (which you already did) so that we finish this migration activity and you can work on the failure.
As a summary, we want to avoid running anything on xenial in Stein.
[1] https://review.openstack.org/#/c/639990/6
-gmann
freezer - not able to connect elasticsearch service
- https://review.openstack.org/#/c/642610/ https://review.openstack.org/#/c/642607/ https://review.openstack.org/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed job as n-v so that this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy jobs migration. Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which will help us to drop the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets their gate. Detailed steps and testing pacthes example are in this etherpad- https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has overridden the nodeset from parent job) Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to bionic which are defined in - https://review.openstack.org/#/c/639018/ Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Thanks all who volunteer for their project and testing the jobs on bionic.
Reminder for all other projects to plan for this activity. Only 5-6 projects have started the testing. I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any clarification.
Thanks.
-gmann
---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > Hi Everyone, > > During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to Bionic. [1]. > > But that did not move all gate job to Bionic as there are a large number of jobs are still the legacy job. All the legacy job still use Xenial as nodeset. > As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic - https://governance.openstack.org/tc/reference/runtimes/stein.html > This is something we discussed TC meeting as one of the Stein items to finish in openstack community[2]. > > I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have created the > Etherpad to include more details and status check for each project gate- https://etherpad.openstack.org/p/legacy-job-bionic > > The approach is the same as we did for previous migration for devstack jobs. > 1. Push patch which migrates the legacy base jobs to bionic - 1. https://review.openstack.org/#/c/639096/ > 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and confirm the status > of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 > 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own. > > Please add yourself as your project volunteer in etherpad and update the status accordingly. I am tagging [ptl] in this thread so that they can assign > someone from their team if no volunteer. > > Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the jobs which I feel enough for each project. > let me know if more time is needed, we can adjust the same. The goal is to finish this activity before Stein release. > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... > https://etherpad.openstack.org/p/devstack-bionic > > [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html > > > -gmann > > >
On Fri, 2019-03-15 at 16:21 -0500, Ghanshyam Mann wrote:
---- On Thu, 14 Mar 2019 23:37:46 -0500 Takashi Yamamoto yamamoto@midokura.com wrote ----
On Wed, Mar 13, 2019 at 2:25 PM Ghanshyam Mann gmann@ghanshyammann.com wrote:
---- On Mon, 11 Mar 2019 22:04:55 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
---- On Mon, 11 Mar 2019 01:03:51 -0500 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- Updates:
I pushed few of the remaining projects testing patch today, below is the latest status. Request you to check your project testing patch and take appropriate action. March 13th is deadline to merge the base legacy jobs migration patch.
[...]
Updates:
I checked the project which are passing their gate on Bionic and which are failed (have not checked the n-v or
experimental jobs)
*FAILING PROJECTS: NEED ACTION* nova - testing with tls-proxy disable - https://review.openstack.org/#/c/639017/ cinder - https://bugs.launchpad.net/cinder/+bug/1819819 designate - https://review.openstack.org/#/c/642319/ trove -https://review.openstack.org/#/c/642329/ zaqar - https://review.openstack.org/#/c/642349/ networking-midonet - https://review.openstack.org/#/c/639990/
is there an option to keep xenial jobs? midonet (the backend networking-midonet talks to) doesn't support bionic yet. https://midonet.atlassian.net/browse/MNA-1344
We really want to avoid that. Main idea if doing this migration at community level is that all the OpenStack projects which are integrated with each other at some level can move and be tested on latest Distro. So that they do not break each other.
In your case, neutron is integrated projects which is or going to be running/tested on Bionic and leaving networking-midonet on xenial can leads to integrated issues at some time. As I have proposed the patch [1], we move the job to Bionic and make it n-v (which you already did) so that we finish this migration activity and you can work on the failure.
As a summary, we want to avoid running anything on xenial in Stein.
im not that familar with midonet but perhaps if it does not have bionic support those jobs could be swapped over to centos 7? i would personally avoid the fedora image but devstack and centos 7 are much happier then they used to be and that would atleast allow project to avoid using xenial with stien. centos 8 will take a while as to exist as RHEL 8 is not even released yet and centos typically trails major rhel release for a few months so CentOS 7 will be the latest release for stien and likely train. in the past centos had more quirks with devstack then ubuntu but for the last year or so i honestly have had equal sucess using bionic or centos 7 so that could be a good path forward for project without bionic support for there dependencies and it effectively just needs the nodeset to be chaged in the job definition.
by the way i may have accidentally broke python 2 support when i enabled python3 on centos last week... its being fix by https://review.openstack.org/#/c/643299/ but on the plus side python3.6 works well on centos too. sorry about :) thats what i get for not testing with python 2 anymore but im honetst plesently suprised by how far centos support has come over the last 18 months so if nothing else it proably worth a patch to just see if the jobs will just run on centos.
a little tangential to the topic the only other thing i will say is while there are docs for how to use the new style jobs i find are still a little confusing but once you figure out how to create non legacy jobs they are actully pretty strait forworad.
what you would have done in a devstackgaterc file in repo https://review.openstack.org/#/c/433491/4/devstack/gate/devstackgaterc port pretty cleanly to a zuul job definition https://review.seanmooney.info/c/test/+/1063/23/.zuul.yaml and a pre_test_hook.sh file https://review.openstack.org/#/c/433491/4/devstack/gate/pre_test_hook.sh can eaisily be ported to an ansible script https://review.seanmooney.info/c/test/+/1063/23/playbooks/dpdk.yaml
although if you like you can still leave the sript in the repo and just run it i only ported the pre_test_hook to ansible because i had not merged the experimental ovs-dpdk job into the networking-ovs-dpdk repo so it was simpler to port now then later.
there is a bit of a learning curve to porting jobs form legacy to non legacy but if you can set aside a day or so you can learn enough ansible and read the required playbook to figure it out. the hardest part honestly was just getting a couple of uninterrupted hours to sit down an do it. I honestly think it worth spending the time to do it as it makes tweaking the jobs significantly simpler in the long run.
[1] https://review.openstack.org/#/c/639990/6
-gmann
freezer - not able to connect elasticsearch service
rg/#/c/642608/ https://review.openstack.org/#/c/642605/
As Next step for all the projects listed above, either check if you can fix the failure otherwise make the failed
job as n-v so that
this migration which is going to happen tomorrow would not block your gate.
*Testing not started*: 4 Loci Openstack-Helm Rally
-gmann
*Testing Started *: 36 projects QA Nova Placement Cinder Glance Keystone Neutron Ironic Manila Designate Monasca Sahara Vitrage Congress Heat Kuryr Mistral: OK Murano Octavia: OK Trove Watcher Oslo Barbican Blazar Ec2-Api Cyborg Karbor Magnum Masakari Telemetry Zaqar Zun kolla Requirements Tricircle Freezer
#OK. no legacy xenial jobs so no action required Qinling Searchlight Senlin Solum Storlets Winstackers Adjutant Release Management I18n Horizon Documentation Cloudkitty Packaging-Rpm Powervmstackers Chef Openstack Tripleo Openstack Charms Openstackclient Openstacksdk Puppet Openstack Openstackansible Swift
*Testing not started*: 4 Loci Openstack-Helm Rally Tacker
-gmann
---- On Thu, 07 Mar 2019 20:34:57 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ----
Updates on planning and dadlines: Legacy jobs to bionic.
I today TC ad hoc meeting on python3 transition & planning[1], we discussed about next steps on legacy
jobs migration.
Also you have seen the Zane's email about the agreement of moving the legacy jobs in Stein [2] which
will help us to drop
the py35 testing.
NEW DEADLINE: March 13th, we will merge the base pacthes which move legacy base jobs nodeset to bionic. ~70% of the project legacy jobs which depends on their parent job nodeset will start running on
bionic.
DO NOT BLOCK GATE: After deadline (when base nodeset is bionic) if any of the your legacy jobs are
failing then, make those jobs non voting and keep working on fix.
How to test your project's legacy jobs : I have updated the base patches which move the legacy base jobs to bionic and ready for projects to tets
their gate.
Detailed steps and testing pacthes example are in this etherpad-
https://etherpad.openstack.org/p/legacy-job-bionic
For quick reference, I am writting the steps to test your legacy jobs on bionic:
Step 1. Push the testing DNM patch on your project owned repos with:
- Depends-On: https://review.openstack.org/#/c/641886/
-Remove or change the nodeset to Bionic from your repo owned legacy jobs (if any of the legacy jobs has
overridden the nodeset from parent job)
Example: https://review.openstack.org/#/c/639017
Step 2. If you have any legacy job not derived from base jobs then, you need to migrate their nodeset to
bionic which are defined in - https://review.openstack.org/#/c/639018/
Example - https://review.openstack.org/#/c/639361/
Step3. if any of the job start failing on bionic then, either fix it before deadline of March 13th or
make failing job as n-v and fix later.
Feel free to ping me for any query or help.
[1] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-0... [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.htm...
-gmann
---- On Thu, 07 Mar 2019 10:55:50 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > Thanks all who volunteer for their project and testing the jobs on bionic. > > Reminder for all other projects to plan for this activity. Only 5-6 projects have started the
testing.
> I request PTL to plan someone for this activity. Feel free to ping me on #openstack-qa for any
clarification.
> > Thanks. > > -gmann > > ---- On Mon, 25 Feb 2019 09:05:47 -0600 Ghanshyam Mann gmann@ghanshyammann.com wrote ---- > > Hi Everyone, > > > > During Dec/Jan month, we have migrated the devstack jobs (zuulv3 native jobs) from Xenial to
Bionic. [1].
> > > > But that did not move all gate job to Bionic as there are a large number of jobs are still the
legacy job. All the legacy job still use Xenial as nodeset.
> > As per the decided runtime for Stein, we need to test everything on OpenStack CI/CD on Bionic -
https://governance.openstack.org/tc/reference/runtimes/stein.html
> > This is something we discussed TC meeting as one of the Stein items to finish in openstack
community[2].
> > > > I am starting this ML thread to coordinate the work to move all the legacy jobs to Bionic. I have
created the
> > Etherpad to include more details and status check for each project gate-
https://etherpad.openstack.org/p/legacy-job-bionic
> > > > The approach is the same as we did for previous migration for devstack jobs. > > 1. Push patch which migrates the legacy base jobs to bionic - 1.
https://review.openstack.org/#/c/639096/
> > 2. Each Project team, add the testing patch with Depends-on on base patch (example 639096) and
confirm the status
> > of their gate on etherpad - Example: https://review.openstack.org/#/c/639017 > > 3. Project jobs not using base job as parent, need to migrate their legacy job to bionic by own. > > > > Please add yourself as your project volunteer in etherpad and update the status accordingly. I am
tagging [ptl] in this thread so that they can assign
> > someone from their team if no volunteer. > > > > Deadline: 1st April to merge the base legacy job to bionic. That gives around 1 month to test the
jobs which I feel enough for each project.
> > let me know if more time is needed, we can adjust the same. The goal is to finish this activity
before Stein release.
> > > > > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000837.... > > https://etherpad.openstack.org/p/devstack-bionic > > > > [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-02-07-14.00.html > > > > > > -gmann > > > > > > > >
participants (10)
-
Doug Hellmann
-
Ghanshyam Mann
-
Graham Hayes
-
Matt Riedemann
-
melanie witt
-
Michael Johnson
-
Sean McGinnis
-
Sean Mooney
-
Slawomir Kaplonski
-
Takashi Yamamoto