Open Stack

On Tue, 2022-02-22 at 10:51 +0100, Thomas Goirand wrote:
> On 2/21/22 14:32, Sean Mooney wrote:
> > On Mon, 2022-02-21 at 13:27 +0000, Sean Mooney wrote:
> > > On Mon, 2022-02-21 at 14:10 +0100, Thomas Goirand wrote:
> > > > On 2/18/22 18:33, Brian Rosmaita wrote:
> > > > > On 2/18/22 11:42 AM, Thomas Goirand wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > I've been quite surprised to see $topic. Most requirements for
> > > > > > os-brick are now to be found in Yoga only. I've asked why, and I've
> > > > > > been told that the reason is because "we've been testing with the new
> > > > > > versions only for the number of months".
> > > > > > 
> > > > > > Well, the reason why someone would increase a minimum requirement
> > > > > > *must* be only because of the need of a new feature, and should be
> > > > > > treated with care. Otherwise, this makes upgrading very dangerous and
> > > > > > annoying to do. As much as possible, I would strongly recommend that
> > > > > > any dependency in Stable can be found in Stable-1 (only when a new
> > > > > > feature is mandatory, then it's ok-ish to require that new version,
> > > > > > though I would advocate for a possible fallback if that's not too
> > > > > > complex to write).
> > > > > > 
> > > > > > Now, if that's the path we're taking, all is going backward 5 years
> > > > > > ago, and then my thinking is: we must reintroduce lower-constraints
> > > > > > testing ASAP.
> > > > > > 
> > > > > > Your thoughts everyone?
> > > > > 
> > > > > It would be nice to have clear guidance on this.  I tried to get
> > > > > pre-release comments about what I planned to do with os-brick, but maybe
> > > > > I didn't allow enough time or had an unclear subject line:
> > > > > 
> > > > > http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027192.html
> > > > 
> > > > I saw the message, but reading it, it wasn't clear enough what the
> > > > intention was, and TBH, I probably read the patch too quickly.
> > > > 
> > > > > My concern about keeping the minima low is that what we're actually
> > > > > testing with in the gate is pretty much whatever's in upper constraints.
> > > > > The previous lower-constraint job basically just checked to see if you
> > > > > could co-install the minimum versions of dependencies and run pass unit
> > > > > tests within a single project, which doesn't seem very realistic.
> > > > 
> > > > It was catching API / ABI breakage, which was the intention.
> > > > 
> > > > > In any case, it would be better to have an openstack-wide policy so we
> > > > > can raise minima in a coordinated way, rather than me forcing everyone
> > > > > who uses os-brick to do whatever I do.
> > > > 
> > > > That's what the global-requirements were supposed to be about, however,
> > > > as far as the CI is concerned, now only the upper-constraints.txt are in
> > > > use, resulting in now useless requirements.txt.
> > > > 
> > > > I very much see a huge regression here, compared to what we had maybe 2
> > > > years ago.
> > > > 
> > > > Indeed, we need to have a (more) global thinking OpenStack wide about
> > > > what we should do. I still believe that all project should be able as
> > > > much as possible to run with all the requirements from stable -1, to
> > > > ease upgrading. This way, it's possible to:
> > > > 1/ upgrade all services without updating the libs
> > > > 2/ then upgrade the libs
> > > > 3/ and finally restart all services.
> > > much of the feature development actully assume you upgade the libs first
> > > then upgrade the service
> > > then restart it.
> > > 
> > > doign the libs second make the code in the consuming porject much more difficult to write
> > > as we need to have fallbacks but upgrading the libs outside of a breaking major version should
> > > not break exisign users.
> > > 
> > > supporting the version of n-1 libs might be worth considering instaed of our older lower constraits aproch
> > > but im really not sure we shoudl expect the service in general to be upgraded first. i can kind of see
> > > why you woudl like that but that woudl increase the burden on client which i dont think we want to do.
> > 
> > by the way the reason i prefer the idea of upgrading the lib first is os-vif and all the other lib projects
> > already have jobs that install the master verion of the lib with the current release of the projects that use them
> > i.e. os-vif is used by nova and neutron so we have a devstack job that ensure we can still work with the current
> > master version of nova and neutron. ideally that means we shoudl never break them with a new release.
> 
> Oh, this makes me think that what I wrote is not completely accurate. 
> While IMO, supporting the stable -1 version of a lib like os-vif and 
> os-brick is important, a lib like neutron-lib of course, is bound to a 
> specific release of OpenStack and the matching Neutron API. So IMO, the 
> reasoning doesn't apply for something like neutron-lib, which contains 
> the internal description of the Neutron API. This however, may make 
> things difficult to upgrade when having Octavia running on the same 
> server as Neutron, but I currently don't see any workaround. Maybe the 
> solution is only to make sure that neutron-lib doesn't break the stable 
> -1 version of Octavia (for example). So in this case, it's more up to 
> neutron-lib to be backward compatible.
> 
> Hoping what I wrote makes sense,
> Cheers,
ya it does we might need to test diffrently for release independent libs vs release coupled libs.
one of the other issue we have is deciding when we bump a dep

for example we are adding support for lightos storage backend to os-brick nova and cinder.
now if you are using lightos you will need a new verion of os-brick but if you dont nova and cinder technically
dont need the latest version which adds support for it.

we dont have an excelent way to track that today.

bindeps has suppport for tags and requirements.txt or well pbr/setuptools more then requiremetns.txt also have the concept of
filtering requiremetns too based on things like python version or you can make use of extras to supprot things like
pip install nova[lightos] ir we wanted to do that.

maybe it would make sense to start tracking deps for optional backend differently.
right now we need to have the new dep for testing but most deployment will use ceph so lightos does not matter in terms of selecting
a min os-brick dep for nova.

i have made the argument in the past that we shoudl not bump the dep in nova in this case and we shoudl instetad document or track this
differently for the less common backend but we never really came to a good way to test that without explodign the jobs.

the one think i will say however is if we cant depend on havign the requried dep installed it forces us to add
check on the version of the lib installed before using some feature which is not somethign we want to have to do in nova
or in your example octavia and neutron both adding check on the neutorn-lib version.

so right now we bump the min version any time any feature in any backend uses a new feature of a lib.
in many cases an older version will work provided you dont use that backend leadign to what feels like an artifically high
version requirement.
> 
> Thomas Goirand (zigo)
>