[TripleO] centos9 jobs only for master, centos 8 & 9 for wallaby
Hello TripleO o/
During the last period the tripleo-ci team has been working on getting tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9 exclusively (since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even between changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby on 8? For example, the coming undercloud-upgrade-ffu job will be train 8 to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely). ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone (no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on getting tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9 exclusively (since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even between changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby on 8? For example, the coming undercloud-upgrade-ffu job will be train 8 to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone (no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on getting tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9 exclusively (since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even between changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby on 8? For example, the coming undercloud-upgrade-ffu job will be train 8 to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone (no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be removed, across stable branches. Does this mean up to and including train? It was my understanding that everything older than wallaby will remain as it is.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on getting tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9 exclusively (since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even between changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby on 8? For example, the coming undercloud-upgrade-ffu job will be train 8 to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be
counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services
is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone (no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be removed, across stable branches. Does this mean up to and including train? It was my understanding that everything older than wallaby will remain as it is.
This discussion is exclusively about wallaby - older stable branches will be unaffected.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on getting tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9 exclusively (since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even between changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby on 8? For example, the coming undercloud-upgrade-ffu job will be train 8 to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone (no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be removed, across stable branches. Does this mean up to and including train? It was my understanding that everything older than wallaby will remain as it is.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on getting tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9 exclusively (since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even between changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby on 8? For example, the coming undercloud-upgrade-ffu job will be train 8 to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone (no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
Great, Thanks for the clarification, it had me worried for a bit.
On Wed, Mar 2, 2022 at 9:05 AM Marios Andreou marios@redhat.com wrote:
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be
removed, across stable branches.
Does this mean up to and including train? It was my understanding that everything older than wallaby will remain
as it is.
This discussion is exclusively about wallaby - older stable branches will be unaffected.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me
wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on
getting
tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9
exclusively
(since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges)
we
will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even
between
changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby
on
8? For example, the coming undercloud-upgrade-ffu job will be train
8
to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be
counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack
services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla
standalone
(no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be
removed, across stable branches.
Does this mean up to and including train? It was my understanding that everything older than wallaby will remain
as it is.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me
wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on
getting
tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9
exclusively
(since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges)
we
will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even
between
changes depending on the files touched - but generally there will be duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8 Wallaby, with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby
on
8? For example, the coming undercloud-upgrade-ffu job will be train
8
to wallaby 8. In which case we definitely need to keep at least some subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would be
counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack
services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla
standalone
(no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so please join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
Just wanted to add that for Ceph integration in TripleO, these are the important Ceph jobs which run when THT is patched in the main branch:
tripleo-ci-centos-9-scenario001-standalone tripleo-ci-centos-9-scenario004-standalone
If the Wallaby branch only had centos-9 versions of the above and no centos-8 versions of the above, then I don't see that as a problem.
John
On Wed, Mar 2, 2022 at 3:08 AM Jiri Podivin jpodivin@redhat.com wrote:
Great, Thanks for the clarification, it had me worried for a bit.
On Wed, Mar 2, 2022 at 9:05 AM Marios Andreou marios@redhat.com wrote:
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for
me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be
removed, across stable branches.
Does this mean up to and including train? It was my understanding that everything older than wallaby will remain
as it is.
This discussion is exclusively about wallaby - older stable branches will be unaffected.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com
wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me
wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com
wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on
getting
tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9
exclusively
(since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132
merges) we
will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even
between
changes depending on the files touched - but generally there will
be
duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8
Wallaby,
with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby
on
8? For example, the coming undercloud-upgrade-ffu job will be
train 8
to wallaby 8. In which case we definitely need to keep at least
some
subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would
be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack
services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is
8
but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla
standalone
(no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so
please
join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for
me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be
removed, across stable branches.
Does this mean up to and including train? It was my understanding that everything older than wallaby will remain
as it is.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com
wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me
wrote:
Hi folks, response in-line.
On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com
wrote:
Hello TripleO o/
During the last period the tripleo-ci team has been working on
getting
tripleo CI jobs to run centos-9.
On master branches across all TripleO repos we now run C9
exclusively
(since [1]).
On Wallaby branches, after the work at [2] (in particular after https://review.opendev.org/c/openstack/tripleo-ci/+/830132
merges) we
will run both C8 and C9 jobs. I started an etherpad at [3] thinking this would be a PTG topic, but I think we'll want to have this conversation way before then. There is a tripleo-heat-templates example in the etherpad that shows what we can expect from the job runs (note the exact jobs that run will vary per repo and even
between
changes depending on the files touched - but generally there will
be
duplication between 8 and 9 jobs).
The current proposal is that we keep a minimal subset on C8
Wallaby,
with C9 wallaby having the full set of jobs.
Two factors that will affect our decisions are (there may be more?)
i) Upgrades requirements - are we supporting upgrading to Wallaby
on
8? For example, the coming undercloud-upgrade-ffu job will be
train 8
to wallaby 8. In which case we definitely need to keep at least
some
subset of 8 jobs (and can't entertain the removal of 8 from Wallaby completely).
I think trying to keep up with the moving target of 8-stream would
be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack
services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is
8
but this will soon switch.
For the wallaby c8 'subset of jobs' e.g. multinode, vanilla
standalone
(no scenarios? some subset of them?), undercloud-ffu, minor update.
This is just to start the conversation so please reply if you have thoughts or comments about any of the above.
We are planning to discuss this in the coming tripleo-ci community call this coming Tuesday at 1330 UTC - meeting link at [4] so
please
join us if you can and would like to participate,
regards, marios
[1] https://review.opendev.org/q/topic:c8_teardown_master [2] https://review.opendev.org/q/topic:c9_wallaby_gates [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 [4] https://meet.google.com/bqx-xwht-wky
On Wed, Mar 2, 2022 at 3:09 PM John Fulton johfulto@redhat.com wrote:
Just wanted to add that for Ceph integration in TripleO, these are the important Ceph jobs which run when THT is patched in the main branch:
tripleo-ci-centos-9-scenario001-standalone tripleo-ci-centos-9-scenario004-standalone
If the Wallaby branch only had centos-9 versions of the above and no centos-8 versions of the above, then I don't see that as a problem.
ack thanks for joining the tripleo-ci community call yesterday and for confirming here
Yes we will continue to have the centos-9 versions of those jobs triggering as before (i.e. same files: matches etc)
regards, marios
John
On Wed, Mar 2, 2022 at 3:08 AM Jiri Podivin jpodivin@redhat.com wrote:
Great, Thanks for the clarification, it had me worried for a bit.
On Wed, Mar 2, 2022 at 9:05 AM Marios Andreou marios@redhat.com wrote:
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be removed, across stable branches. Does this mean up to and including train? It was my understanding that everything older than wallaby will remain as it is.
This discussion is exclusively about wallaby - older stable branches will be unaffected.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me wrote:
Hi folks, response in-line.
> On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote: > > Hello TripleO o/ > > During the last period the tripleo-ci team has been working on getting > tripleo CI jobs to run centos-9. > > On master branches across all TripleO repos we now run C9 exclusively > (since [1]). > > On Wallaby branches, after the work at [2] (in particular after > https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we > will run both C8 and C9 jobs. I started an etherpad at [3] thinking > this would be a PTG topic, but I think we'll want to have this > conversation way before then. There is a tripleo-heat-templates > example in the etherpad that shows what we can expect from the job > runs (note the exact jobs that run will vary per repo and even between > changes depending on the files touched - but generally there will be > duplication between 8 and 9 jobs). > > The current proposal is that we keep a minimal subset on C8 Wallaby, > with C9 wallaby having the full set of jobs. > > Two factors that will affect our decisions are (there may be more?) > > i) Upgrades requirements - are we supporting upgrading to Wallaby on > 8? For example, the coming undercloud-upgrade-ffu job will be train 8 > to wallaby 8. In which case we definitely need to keep at least some > subset of 8 jobs (and can't entertain the removal of 8 from Wallaby > completely).
I think trying to keep up with the moving target of 8-stream would be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
> ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 > but this will soon switch. > > For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone > (no scenarios? some subset of them?), undercloud-ffu, minor update. > > This is just to start the conversation so please reply if you have > thoughts or comments about any of the above. > > We are planning to discuss this in the coming tripleo-ci community > call this coming Tuesday at 1330 UTC - meeting link at [4] so please > join us if you can and would like to participate, > > regards, marios > > [1] https://review.opendev.org/q/topic:c8_teardown_master > [2] https://review.opendev.org/q/topic:c9_wallaby_gates > [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 > [4] https://meet.google.com/bqx-xwht-wky > >
On Wed, Mar 2, 2022 at 9:44 AM Jiri Podivin jpodivin@redhat.com wrote:
Hi, Sorry I'm replying this late, but this line raised some questions for me:
To be clear, our plan is to remove *all* centos8 check/gate jobs,
It seems to imply that all centos8 jobs in both pipelines will be removed, across stable branches. Does this mean up to and including train? It was my understanding that everything older than wallaby will remain as it is.
On Wed, Mar 2, 2022 at 8:04 AM Marios Andreou marios@redhat.com wrote:
On Tue, Mar 1, 2022 at 5:03 PM Jesse Pretorius jesse@odyssey4.me wrote:
Hi folks, response in-line.
> On 24 Feb 2022, at 15:11, Marios Andreou marios@redhat.com wrote: > > Hello TripleO o/ > > During the last period the tripleo-ci team has been working on getting > tripleo CI jobs to run centos-9. > > On master branches across all TripleO repos we now run C9 exclusively > (since [1]). > > On Wallaby branches, after the work at [2] (in particular after > https://review.opendev.org/c/openstack/tripleo-ci/+/830132 merges) we > will run both C8 and C9 jobs. I started an etherpad at [3] thinking > this would be a PTG topic, but I think we'll want to have this > conversation way before then. There is a tripleo-heat-templates > example in the etherpad that shows what we can expect from the job > runs (note the exact jobs that run will vary per repo and even between > changes depending on the files touched - but generally there will be > duplication between 8 and 9 jobs). > > The current proposal is that we keep a minimal subset on C8 Wallaby, > with C9 wallaby having the full set of jobs. > > Two factors that will affect our decisions are (there may be more?) > > i) Upgrades requirements - are we supporting upgrading to Wallaby on > 8? For example, the coming undercloud-upgrade-ffu job will be train 8 > to wallaby 8. In which case we definitely need to keep at least some > subset of 8 jobs (and can't entertain the removal of 8 from Wallaby > completely).
I think trying to keep up with the moving target of 8-stream would be counterproductive given that we are pinned downstream to RHEL 8.4 for Train. We do have a pending patch to merge a job for 8-Stream/Train->8-Stream Wallaby for the Undercloud only, which would be nice to keep but if it becomes troublesome then I’d suggest we remove it. This job adds some value to us (and has added quite a bit already in preparing it and making it pass), but if 8-stream starts making it fail, or it starts failing due to database migrations then we’ll have to shift that job downstream and deal with the consequences of only ever finding upgrade bugs post-merge.
It is well known that a Fast-Forward Upgrade (FFU) of OpenStack services is not supported upstream, so we can not expect any upstream jobs to reliability support this process. If/when there is broader support for enabling FFU support officially in OpenStack services, then we can reconsider this position.
Hi Jesse,
thanks for replying here too.
As shaped by the discussions yesterday and for the benefit of others - our current plan is to keep the c8 undercloud-ffu job and in fact that will be the *only* job we will keep on C8 (if standalone ffu upgrade becomes a thing then that will also be 8 only). At least, we'll make a best effort to keep this and we can re-evaluate once we hit blocking issues.
In order to maintain this C8 gate we'll also need to maintain the equivalent periodic version for the integration line. So we'll have a C8 integration line running the undercloud-ffu job and possibly one ovb job (TBD, likely featureset 1).
To be clear, our plan is to remove *all* centos8 check/gate jobs, except the undercloud-ffu. We are not planning to keep any 'base' set of c8 jobs unless someone can make a case for their value.
The tripleo-ci team will start implementing the removal of c8/wallaby jobs once we switch to use c9 for the imports which should happen within the next week or so.
regards, marios
> ii) Are we importing from Wallaby 8 or Wallaby 9? Currently it is 8 > but this will soon switch. > > For the wallaby c8 'subset of jobs' e.g. multinode, vanilla standalone > (no scenarios? some subset of them?), undercloud-ffu, minor update. > > This is just to start the conversation so please reply if you have > thoughts or comments about any of the above. > > We are planning to discuss this in the coming tripleo-ci community > call this coming Tuesday at 1330 UTC - meeting link at [4] so please > join us if you can and would like to participate, > > regards, marios > > [1] https://review.opendev.org/q/topic:c8_teardown_master > [2] https://review.opendev.org/q/topic:c9_wallaby_gates > [3] https://etherpad.opendev.org/p/tripleoci-wallaby-centos-8-9 > [4] https://meet.google.com/bqx-xwht-wky > >
participants (4)
-
Jesse Pretorius
-
Jiri Podivin
-
John Fulton
-
Marios Andreou