On Mon, 2022-10-03 at 13:50 -0700, Clark Boylan wrote:

On Mon, Oct 3, 2022, at 1:06 PM, Jay Faulkner wrote:

...

Hey all,

I was attempting to perform a release of all maintained Ironic branches. The long term support branches cut as part of the integrated OpenStack release have been requested via gerrit, as documented ( https://review.opendev.org/c/openstack/releases/+/860125 ).

We have intermediate bugfix releases as well, and I was hoping to get patch releases cut from these as well. As far as I can tell, there is no automation for performing these releases, or an official place to request them. If this is wrong; please correct me and I'm happy to go through the proper process.

We know that the majority of consumers pull these from git directly; but checking pypi release notes, we had over 6000 downloads of the 21.0.0 and 20.2.0 releases during the 2/4 months of the Zed cycle they had respectively been released, so I do think there's value in releasing these even if it might be a small amount of manual effort.

Below is a list of the Ironic projects, and associated bugfix/ branches I'd like to have a patch (bugfix) release cut for:

ironic - bugfix/21.0 - bugfix/20.2 - bugfix/19.0 - bugfix/18.1

ironic-inspector - bugfix/11.0 - bugfix/10.12 - bugfix/10.9 - bugfix/10.7

ironic-python-agent - bugfix/9.0 - bugfix/8.6 - bugfix/8.3 - bugfix/8.1

As mentioned in the openstack-releases IRC channel it seems that the Ironic project doesn't have ACLs to push their own manual tags [0]. It was also mentioned that the release team thought releases off of the bugfix branches would need manual releases. I think the lack of ACLs to do this by the Ironic project means that the release team needs to do it, or we need to modify the ACLs to allow the Ironic team to do the work.

If the release team ends up doing the work, it would probably be a good idea to very explicitly list the branch, commit sha1, and version number for each of the needed releases. This way the release team doesn't have to guess if they are getting it correct when they make and push those tags.

Separately, it seems like some of the intention here is to ensure that users of bugfix branches don't end up with stale installations. Updating the release tooling to handle releases off of these branches or delegating access to the Ironic team seem like an important piece of making that happen. Otherwise the overhead for doing this will be large enough that it is unlikely to happen often enough. Unfortunately, I don't know what is currently missing in the tooling to make that possible.

[0] https://opendev.org/openstack/project-config/src/branch/master/gerrit/acls/o...

its litrally been about 7 or 8 years since i did this but i had configred networkign-ovs-dpdk so that we could push sgined tags teh networking-ovs-dpdk-release group has the required acls to be able to create branches and tags. [access "refs/heads/*"] create = group networking-ovs-dpdk-release allows creating branches and enable branch creation and [access "refs/tags/*"] createSignedTag = group networking-ovs-dpdk-release enables pushing signed tags which used to get mirrored to github as well. this wont auto push the content to pypi i sued to also build the package locally an push it manually but that would enabel ironich to actully tag and push the content if they are added to the pypi repo as well. ideally however i think it woudl be better long term to just do this via the releases repo. im not entirly sure what prevent you just adding a new bugfix branch and sha there https://github.com/openstack/releases/blob/master/deliverables/zed/ironic.ya... you can have one patch taht update all the bug fix branches acroos multipel release and then the release team just need to review one path. presumable this wont happen more frequently then say 1 a quater or once a month so that is proably doable vai the normal release process.

...

Thanks, Jay Faulkner

On Wed, Oct 5, 2022 at 7:18 AM Thierry Carrez <thierry@openinfra.dev> wrote:

Clark Boylan wrote:

...

[...] If the release team ends up doing the work, it would probably be a good idea to very explicitly list the branch, commit sha1, and version number for each of the needed releases. This way the release team doesn't have to guess if they are getting it correct when they make and push those tags.

Yes please.

I'm cleaning up some CI right now; I'll make sure we get this information to the list soon so we can give it a shot :).

...

Separately, it seems like some of the intention here is to ensure that users of bugfix branches don't end up with stale installations. Updating the release tooling to handle releases off of these branches or delegating access to the Ironic team seem like an important piece of making that happen. Otherwise the overhead for doing this will be large enough that it is unlikely to happen often enough. Unfortunately, I don't know what is currently missing in the tooling to make that possible.

I'd say it's an unknown and the current release team members may not have bandwidth to explore what releasing on bugfix branches using a patch to openstack/releases could look like. Avoiding collisions between "normal" stable branch point updates and those bugfix branch point releases sounds tricky at best.

I'm hoping this won't be an issue. Ironic policy (and it seems to be true in practice), says any release from master (including bugfix/x branches) must bump either the major or minor release number ( https://specs.openstack.org/openstack/ironic-specs/specs/15.1/new-release-mo... ). Thanks Thierry and Clark, I'll get you the information you all need to move forward soon! -Jay Faulkner

It sounds like no releases have ever been made from any of the

"bugfix" branches going back over two years since their creation. Is this a change in how the Ironic team views those branches, or was the intention always to tag releases on them but nobody had gotten around to it?

The primary consumer historically of bugfix branches have been downstream packagers of Ironic -- e.g. openshift. I'm pursuing creating releases of these because after doing some research, we have customers consuming these releases from pypi (6000 downloads of our Zed-cycle releases in 4 months). I do not want those deployers, consuming upstream artifacts, to miss out on backported bugfixes and patches that would be provided in a vendor release artifact. There is no urgency behind getting the bugfix releases made, but the current situation where the Ironic community maintains additional branches for longer term support but does not provide stable releases for customers puiling upstream release artifacts is not one I wish to perpetuate. -- Jay Faulkner

That makes sense, but it's worth noting what you have now is not entirely dissimilar to how "extended maintenance" works for our official stable branches as well (continuing to merge some backported fixes, but not tagging any further point releases).

Yeah, I understand. This is all working towards a revision of Ironic release policy to ensure it's documented when/how these releases are supported. Right now we create bugfix releases as months 2 and 4 into the cycle, with no indication of support length. In fact; there are leftover bugfix/[] branches that are not maintained and have not yet been retired. Posts like this (and my efforts to get releases done) is part of my fact-finding for trying to ensure how Ironic manages releases is well documented and understood. -Jay Faulkner