[all] [requirements] Artificially inflated requirements in os-brick
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
Cheers,
Thomas Goirand (zigo)
P.S: Please note that this is absolutely not about os-brick, I'm not pointing my finger to any team, it's a global thinking for the whole OpenStack project.
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
Cheers,
Thomas Goirand (zigo)
P.S: Please note that this is absolutely not about os-brick, I'm not pointing my finger to any team, it's a global thinking for the whole OpenStack project.
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
without troubles with co-existing services. Requiring a lib from the current release means that potentially, during the upgrade, an older service may run with a library from oldstable+1, which that service doesn't know about (yet). If that's not always possible, it's of course IMO ok to allow exceptions, if they are under control, and if we make sure there's no backward breakage (which can only be checked manually).
Cheers,
Thomas Goirand (zigo)
On Mon, 2022-02-21 at 14:10 +0100, Thomas Goirand wrote:
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
much of the feature development actully assume you upgade the libs first then upgrade the service then restart it.
doign the libs second make the code in the consuming porject much more difficult to write as we need to have fallbacks but upgrading the libs outside of a breaking major version should not break exisign users.
supporting the version of n-1 libs might be worth considering instaed of our older lower constraits aproch but im really not sure we shoudl expect the service in general to be upgraded first. i can kind of see why you woudl like that but that woudl increase the burden on client which i dont think we want to do.
without troubles with co-existing services. Requiring a lib from the current release means that potentially, during the upgrade, an older service may run with a library from oldstable+1, which that service doesn't know about (yet). If that's not always possible, it's of course IMO ok to allow exceptions, if they are under control, and if we make sure there's no backward breakage (which can only be checked manually).
Cheers,
Thomas Goirand (zigo)
On Mon, 2022-02-21 at 13:27 +0000, Sean Mooney wrote:
On Mon, 2022-02-21 at 14:10 +0100, Thomas Goirand wrote:
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
much of the feature development actully assume you upgade the libs first then upgrade the service then restart it.
doign the libs second make the code in the consuming porject much more difficult to write as we need to have fallbacks but upgrading the libs outside of a breaking major version should not break exisign users.
supporting the version of n-1 libs might be worth considering instaed of our older lower constraits aproch but im really not sure we shoudl expect the service in general to be upgraded first. i can kind of see why you woudl like that but that woudl increase the burden on client which i dont think we want to do.
by the way the reason i prefer the idea of upgrading the lib first is os-vif and all the other lib projects already have jobs that install the master verion of the lib with the current release of the projects that use them i.e. os-vif is used by nova and neutron so we have a devstack job that ensure we can still work with the current master version of nova and neutron. ideally that means we shoudl never break them with a new release.
extending that to n-1 would be simpler to do then inverting the upgrade worklow to upgrade the service before the libs. our current testing today assuems the libs will be upgraded first so extending that approch to test more then just the current release is less invaisive.
without troubles with co-existing services. Requiring a lib from the current release means that potentially, during the upgrade, an older service may run with a library from oldstable+1, which that service doesn't know about (yet). If that's not always possible, it's of course IMO ok to allow exceptions, if they are under control, and if we make sure there's no backward breakage (which can only be checked manually).
Cheers,
Thomas Goirand (zigo)
On 2/21/22 14:32, Sean Mooney wrote:
On Mon, 2022-02-21 at 13:27 +0000, Sean Mooney wrote:
On Mon, 2022-02-21 at 14:10 +0100, Thomas Goirand wrote:
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
much of the feature development actully assume you upgade the libs first then upgrade the service then restart it.
doign the libs second make the code in the consuming porject much more difficult to write as we need to have fallbacks but upgrading the libs outside of a breaking major version should not break exisign users.
supporting the version of n-1 libs might be worth considering instaed of our older lower constraits aproch but im really not sure we shoudl expect the service in general to be upgraded first. i can kind of see why you woudl like that but that woudl increase the burden on client which i dont think we want to do.
by the way the reason i prefer the idea of upgrading the lib first is os-vif and all the other lib projects already have jobs that install the master verion of the lib with the current release of the projects that use them i.e. os-vif is used by nova and neutron so we have a devstack job that ensure we can still work with the current master version of nova and neutron. ideally that means we shoudl never break them with a new release.
Oh, this makes me think that what I wrote is not completely accurate. While IMO, supporting the stable -1 version of a lib like os-vif and os-brick is important, a lib like neutron-lib of course, is bound to a specific release of OpenStack and the matching Neutron API. So IMO, the reasoning doesn't apply for something like neutron-lib, which contains the internal description of the Neutron API. This however, may make things difficult to upgrade when having Octavia running on the same server as Neutron, but I currently don't see any workaround. Maybe the solution is only to make sure that neutron-lib doesn't break the stable -1 version of Octavia (for example). So in this case, it's more up to neutron-lib to be backward compatible.
Hoping what I wrote makes sense, Cheers,
Thomas Goirand (zigo)
On Tue, 2022-02-22 at 10:51 +0100, Thomas Goirand wrote:
On 2/21/22 14:32, Sean Mooney wrote:
On Mon, 2022-02-21 at 13:27 +0000, Sean Mooney wrote:
On Mon, 2022-02-21 at 14:10 +0100, Thomas Goirand wrote:
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
much of the feature development actully assume you upgade the libs first then upgrade the service then restart it.
doign the libs second make the code in the consuming porject much more difficult to write as we need to have fallbacks but upgrading the libs outside of a breaking major version should not break exisign users.
supporting the version of n-1 libs might be worth considering instaed of our older lower constraits aproch but im really not sure we shoudl expect the service in general to be upgraded first. i can kind of see why you woudl like that but that woudl increase the burden on client which i dont think we want to do.
by the way the reason i prefer the idea of upgrading the lib first is os-vif and all the other lib projects already have jobs that install the master verion of the lib with the current release of the projects that use them i.e. os-vif is used by nova and neutron so we have a devstack job that ensure we can still work with the current master version of nova and neutron. ideally that means we shoudl never break them with a new release.
Oh, this makes me think that what I wrote is not completely accurate. While IMO, supporting the stable -1 version of a lib like os-vif and os-brick is important, a lib like neutron-lib of course, is bound to a specific release of OpenStack and the matching Neutron API. So IMO, the reasoning doesn't apply for something like neutron-lib, which contains the internal description of the Neutron API. This however, may make things difficult to upgrade when having Octavia running on the same server as Neutron, but I currently don't see any workaround. Maybe the solution is only to make sure that neutron-lib doesn't break the stable -1 version of Octavia (for example). So in this case, it's more up to neutron-lib to be backward compatible.
Hoping what I wrote makes sense, Cheers,
ya it does we might need to test diffrently for release independent libs vs release coupled libs. one of the other issue we have is deciding when we bump a dep
for example we are adding support for lightos storage backend to os-brick nova and cinder. now if you are using lightos you will need a new verion of os-brick but if you dont nova and cinder technically dont need the latest version which adds support for it.
we dont have an excelent way to track that today.
bindeps has suppport for tags and requirements.txt or well pbr/setuptools more then requiremetns.txt also have the concept of filtering requiremetns too based on things like python version or you can make use of extras to supprot things like pip install nova[lightos] ir we wanted to do that.
maybe it would make sense to start tracking deps for optional backend differently. right now we need to have the new dep for testing but most deployment will use ceph so lightos does not matter in terms of selecting a min os-brick dep for nova.
i have made the argument in the past that we shoudl not bump the dep in nova in this case and we shoudl instetad document or track this differently for the less common backend but we never really came to a good way to test that without explodign the jobs.
the one think i will say however is if we cant depend on havign the requried dep installed it forces us to add check on the version of the lib installed before using some feature which is not somethign we want to have to do in nova or in your example octavia and neutron both adding check on the neutorn-lib version.
so right now we bump the min version any time any feature in any backend uses a new feature of a lib. in many cases an older version will work provided you dont use that backend leadign to what feels like an artifically high version requirement.
Thomas Goirand (zigo)
On 2/21/22 14:27, Sean Mooney wrote:
On Mon, 2022-02-21 at 14:10 +0100, Thomas Goirand wrote:
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
much of the feature development actully assume you upgade the libs first then upgrade the service then restart it.
If we support the version of the libs from stable-1, this means that effectively, we support the workflow I'm talking about.
The way you are describing the upgrade (ie: libs first) simply doesn't work. It would mean services from stable -1 support libs from stable: nobody has a crystal ball, and it's impossible to support the changes we're going to do in 6 months of time. So the only thing we can do is support backward compatibility.
Note that I haven't advocate for *only* supporting libs from stable -1. I think this must be the lowest bar to set. I'd very much prefer if we were doing things right (tm), meaning supporting the lowest possible version. This means one would increase the lower bound of a requirement only when needing a new API not provided by an earlier version of the lib. Though giving the current constraints, maybe testing with the version of the libs from stable -1 would work.
Also, let's keep in mind the current TC proposal (ie: mandate allowing to skip every odd release during upgrades). If it goes forward, then probably we will need to support libs from stable -2 on every odd release of OpenStack.
Cheers,
Thomas Goirand (zigo)
Hi, I just would like to put my 2 cents on the general idea of having something like we had lower-constraints testing (on master in networking we still have l-c job, but not on stable branches). I think generally everybody agrees that such testing can be really helpful for distro builders for example. The issue is that there is nobody to keep these requirement lists continuously under control.
Cheers, Lajos Katona (lajoskatona)
Thomas Goirand zigo@debian.org ezt írta (időpont: 2022. febr. 21., H, 14:17):
On 2/18/22 18:33, Brian Rosmaita wrote:
On 2/18/22 11:42 AM, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
It would be nice to have clear guidance on this. I tried to get pre-release comments about what I planned to do with os-brick, but maybe I didn't allow enough time or had an unclear subject line:
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192....
I saw the message, but reading it, it wasn't clear enough what the intention was, and TBH, I probably read the patch too quickly.
My concern about keeping the minima low is that what we're actually testing with in the gate is pretty much whatever's in upper constraints. The previous lower-constraint job basically just checked to see if you could co-install the minimum versions of dependencies and run pass unit tests within a single project, which doesn't seem very realistic.
It was catching API / ABI breakage, which was the intention.
In any case, it would be better to have an openstack-wide policy so we can raise minima in a coordinated way, rather than me forcing everyone who uses os-brick to do whatever I do.
That's what the global-requirements were supposed to be about, however, as far as the CI is concerned, now only the upper-constraints.txt are in use, resulting in now useless requirements.txt.
I very much see a huge regression here, compared to what we had maybe 2 years ago.
Indeed, we need to have a (more) global thinking OpenStack wide about what we should do. I still believe that all project should be able as much as possible to run with all the requirements from stable -1, to ease upgrading. This way, it's possible to: 1/ upgrade all services without updating the libs 2/ then upgrade the libs 3/ and finally restart all services.
without troubles with co-existing services. Requiring a lib from the current release means that potentially, during the upgrade, an older service may run with a library from oldstable+1, which that service doesn't know about (yet). If that's not always possible, it's of course IMO ok to allow exceptions, if they are under control, and if we make sure there's no backward breakage (which can only be checked manually).
Cheers,
Thomas Goirand (zigo)
On 2/18/22 17:42, Thomas Goirand wrote:
Hi,
I've been quite surprised to see $topic. Most requirements for os-brick are now to be found in Yoga only. I've asked why, and I've been told that the reason is because "we've been testing with the new versions only for the number of months".
Well, the reason why someone would increase a minimum requirement *must* be only because of the need of a new feature, and should be treated with care. Otherwise, this makes upgrading very dangerous and annoying to do. As much as possible, I would strongly recommend that any dependency in Stable can be found in Stable-1 (only when a new feature is mandatory, then it's ok-ish to require that new version, though I would advocate for a possible fallback if that's not too complex to write).
Now, if that's the path we're taking, all is going backward 5 years ago, and then my thinking is: we must reintroduce lower-constraints testing ASAP.
Your thoughts everyone?
To demonstrate what I'm saying (that we got inflated versions for no reasons), I've analysed some of the version bumps by quickly reviewing the matching git commits.
-oslo.concurrency>=4.4.0 # Apache-2.0 +oslo.concurrency>=4.5.0 # Apache-2.0
A single commit: "Add support for non-blocking locks" The lock() function of lockutils gained a blocking=True default param, that one has to set to False to use the new feature. A quick grep in the os-brick code shows only os_brick/initiator/utils.py uses that function, and doesn't have the "False" blocking param.
-oslo.context>=3.1.1 # Apache-2.0 +oslo.context>=3.4.0 # Apache-2.0
Only this commit is relevant: https://review.opendev.org/c/openstack/oslo.context/+/800852
this is a bug fix, which was backported to all branches since Ussuri.
-oslo.log>=4.4.0 # Apache-2.0 +oslo.log>=4.6.1 # Apache-2.0
- One patch that removes the u in u'' strings... - One config option typo fix. - Some "we run on Python3" commits. - More cosmetic fixes...
-oslo.i18n>=5.0.1 # Apache-2.0 +oslo.i18n>=5.1.0 # Apache-2.0
- A six removal patch. - Some "we use python3" patches. - Some more "we are currently in Xena" patches.
At this point I'm stopping the analysis. It's crystal clear that os-brick requirement version bumps are *NOT* beacause os-brick needs any new feature in these libs, and that it's really artificially inflated just because of the thinking that the gate isn't using older versions ... so we don't easlily know about any possible breakage.
Reality checked: I don't see why it would break. However, it'd be nice to have some more automated ways to do such checks.
These were easily analysed, but it would be harder for eg Eventlet to know if os-brick really needs version 0.30.1. However, it should be a way more easy for a developer of os-brick to know if his change is affecting the minimum requirements. IMO it's at this moment that the bump should happen.
I do understand the "we only tests with the higher bounds", however, again, this may affect upgrades, and it's not reflecting any reality.
Once more: this is not against os-brick, but a higher level thinking. Anyone (even someone not contributing in os-brick) is welcome in this discussion, as I'm trying to discuss here what the OpenStack *policy* should be.
I'm now thinking: we should re-do lower constraints checks!!!
Your thoughts anyone?
Cheers,
Thomas Goirand (zigo)
On 2022-02-22 17:56:15 +0100 (+0100), Thomas Goirand wrote: [...]
I'm now thinking: we should re-do lower constraints checks!!!
Your thoughts anyone?
The reason they were dropped is that the current way we install Python library dependencies for testing is to rely on pip, and when pip eventually grew a consistent dependency solver it quickly pointed out that we weren't testing what we thought we were. Back when pip was willing to install mutually-conflicting versions of transitive dependencies, the lower bounds we claimed to test were a convenient fiction.
There is no automated tool available which can find a coherent set of lower bounds; pip is focused on finding the highest available versions which meet specified ranges. One reason such a tool doesn't exist though is that, as you choose earlier and earlier versions of packages, you also effectively travel back in time to older packaging standards with less available data for making appropriate dependency decisions (and you'll eventually hit bedrock in places where nobody was declaring minimum versions so you get foo==0.0.1 for lots of foo). It's this "time travel" aspect which is most problematic, because dependencies are declared within the packages themselves, so there is no way to go back later and update the minimum requirements for an already published version.
It's probably feasible to hand curate, with lots of trial and error, a coherent lower bound set for a project with a moderate number of dependencies, but every time you update it you have to repeat that same cumbersome manual experimentation due to ripple effects from interdependencies between the packages. At OpenStack's collective scale, it borders on impossible.
participants (5)
-
Brian Rosmaita
-
Jeremy Stanley
-
Lajos Katona
-
Sean Mooney
-
Thomas Goirand