Open Stack

On 9/8/21 1:21 AM, Jeremy Stanley wrote:
> On 2021-09-08 00:46:30 +0200 (+0200), Thomas Goirand wrote:
> [...]
>> In fact, I was particularly confuse to realize that something like
>> taskflow was declared release-independent since Wallaby. Now, if
>> I'm told that it is loosely coupled with OpenStack, knowing that
>> it's a key component, I'm not sure what I should do. Should I get
>> taskflow updated to 4.6.2 in Wallaby, so I get bugfixes? Or is it
>> safer to just upgrade it for Xena?
>>
>> I took Taskflow as an example (you can reply to me about Taskflow
>> in particular, but that's not the point I'm trying to make). But
>> it could have been something else (deptcollector? oslo.i18n?).
>> What I'm trying to say is that as a downstream consumer, it's from
>> hard to impossible to know what to do for the past OpenStack
>> releases regarding these release-independent components: should I
>> update them or not? It is unrealistic to hope that one can track
>> what component has been updated to a newer version in the stable
>> release CI.
> 
> Specific examples are great actually, because it at least lets us
> try to formulate a specific answer and then reason about whether a
> similar answer can be generally applied to other related scenarios.
> So let's take taskflow in Wallaby and Xena:
> 
> The closest thing we have to a recommendation for versions of
> "outside" dependencies (which is essentially what we treat
> release-independent deliverables as) is the upper-constraints.txt
> file on each branch of the openstack/requirements repository. In
> the case of taskflow, at the moment the stable/wallaby branch upper
> constraint for taskflow is 4.6.0, which is the version we indicate
> all OpenStack projects should test their stable/wallaby changes
> against. That entry was last updated by
> https://review.opendev.org/773620 which merged to the master branch
> of openstack/requirements in February, well before the coordinated
> Wallaby release.

Ok, so your recommendation is that, for each and every component that is
marked release-independent, I should be checking the global
requirements. I just wrote that this is unrealistic that a human does
such a check.

If you didn't know, in Debian, Michal Arbet (who co-maintains OpenStack
with me together in Debian) wrote this:

https://salsa.debian.org/openstack-team/debian/os-version-checker

This VersionStatus.py checks for what's in
https://releases.openstack.org/, for example in
https://releases.openstack.org/xena/index.html for the Xena upcoming
release.

The result output of the os-version-checker is this page:

http://osbpo.debian.net/deb-status/

Thanks to this page, anyone can know the status of OpenStack packaging
on all the different branches we maintain. If you look at it right now,
you'll notice that we're up-to-date, as we packaged all non-clients and
all clients library, but we didn't package new versions of the puppet
manifests yet (I'm waiting for the RCs to do that).

If you click on the buster-victoria button, and look for Taskflow,
you'll see that it's up-to-date. If you click on bullseye-xena, and
search for Taskflow, you will ... find nothing! That's because taskflow
doesn't appear in https://releases.openstack.org/xena/index.html since
it is release-independent. As a result, I'm left with no other choice
than being very careful and manually check the global-requirements.

Every check that is supposed to be done by a human isn't reliable. It'd
be nice if I had a way to automatically check for this. The thing is,
the global-requirements is full of information, some of it not relevant
to my own use case, and it's hard to check automatically. First of all,
because the Python module names as in PyPi, do not necessarily match the
package names. Also, because some artifacts are not relevant to Debian.
For example, I have no use of many TripleO stuff, and some services are
simply not packaged in Debian.

In an ideal world, we'd have an API to check for all of this, rather
than digging through the release HTML pages.

> It seems like what you're asking for is this:
> 
> https://opendev.org/openstack/requirements/src/branch/stable/wallaby/upper-constraints.txt
> 
> Or rather, that's the closest thing we have to what I think you're
> requesting. If you follow the commit history of that file, you'll
> see recent updates for it on the stable/wallaby branch do consist
> almost entirely of stable point releases of cycle-oriented projects:
> 
> https://opendev.org/openstack/requirements/commits/branch/stable/wallaby/upper-constraints.txt
> 
> Does this help narrow the scope of concern to the point where we can
> better reason about the problems or challenges you're facing?

Unfortunately, while it is part of the answer, due to what I explained
just above, it is very hard to exploit the available data without a
real, long effort on our side to fix this. :/

One solution could probably be for *US* (downstream distros) to write
some kind of API as I just wrote, and make it available for everyone on
releases.openstack.org. Are Red Hat people also interested by such an
effort? Any pointer where we should attempt to propose patches?

Cheers,

Thomas Goirand (zigo)