January 2022 - openstack-discuss - lists.openstack.org

[ops][nova] How to update from N to N+3 (and manage nova services that don't start because they find too old compute nodes...)
by Massimo Sgaravatto 28 Feb '22

28 Feb '22

Dear all When we upgraded our Cloud from Rocky to Train we followed the following procedure: 1) Shutdown of all services on the controller and compute nodes 2) Update from Rocky to Stein of controller (just to do the dbsyncs) 3) Update from Stein to Train of controller 4) Update from Rocky to Train of compute nodes We are trying to do the same to update from Train to Xena, but now there is a problem because nova services on the controller node refuse to start since they find too old compute nodes (this is indeed a new feature, properly documented in the release notes). As a workaround we had to manually modify the "version" field of the compute nodes in the nova.services table. Is it ok, or is there a cleaner way to manage the issue ? Thanks, Massimo

5 10

[all][tc] Relmgt team position on release cadence
by Thierry Carrez 14 Feb '22

14 Feb '22

Hi everyone, The (long) document below reflects the current position of the release management team on a popular question: should the OpenStack release cadence be changed? Please note that we only address the release management / stable branch management facet of the problem. There are other dimensions to take into account (governance, feature deprecation, supported distros...) to get a complete view of the debate. Introduction ------------ The subject of how often OpenStack should be released has been regularly debated in the OpenStack community. OpenStack started with a 3-month release cycle, then switched to 6-month release cycle starting with Diablo. It is often thought of a release management decision, but it is actually a much larger topic: a release cadence is a trade-off between pressure to release more often and pressure to release less often, coming in from a lot of different stakeholders. In OpenStack, it is ultimately a Technical Committee decision. But that decision is informed by the position of a number of stakeholders. This document gives historical context and describes the current release management team position. The current trade-off --------------------- The main pressure to release more often is to make features available to users faster. Developers get a faster feedback loop, hardware vendors ensure software is compatible with their latest products, and users get exciting new features. "Release early, release often" is a best practice in our industry -- we should generally aim at releasing as often as possible. But that is counterbalanced by pressure to release less often. From a development perspective, each release cycle comes with some process overhead. On the integrators side, a new release means packaging and validation work. On the users side, it means pressure to upgrade. To justify that cost, there needs to be enough user-visible benefit (like new features) in a given release. For the last 10 years for OpenStack, that balance has been around six months. Six months let us accumulate enough new development that it was worth upgrading to / integrating the new version, while giving enough time to actually do the work. It also aligned well with Foundation events cadence, allowing to synchronize in-person developer meetings date with start of cycles. What changed ------------ The major recent change affecting this trade-off is that the pace of new development in OpenStack slowed down. The rhythm of changes was divided by 3 between 2015 and 2021, reflecting that OpenStack is now a mature and stable solution, where accessing the latest features is no longer a major driver. That reduces some of the pressure for releasing more often. At the same time, we have more users every day, with larger and larger deployments, and keeping those clusters constantly up to date is an operational challenge. That increases the pressure to release less often. In essence, OpenStack is becoming much more like a LTS distribution than a web browser -- something users like moving slow. Over the past years, project teams also increasingly decoupled individual components from the "coordinated release". More and more components opted for an independent or intermediary-released model, where they can put out releases in the middle of a cycle, making new features available to their users. This increasingly opens up the possibility of a longer "coordinated release" which would still allow development teams to follow "release early, release often" best practices. All that recent evolution means it is (again) time to reconsider if the 6-month cadence is what serves our community best, and in particular if a longer release cadence would not suit us better. The release management team position on the debate -------------------------------------------------- While releasing less often would definitely reduce the load on the release management team, most of the team work being automated, we do not think it should be a major factor in motivating the decision. We should not adjust the cadence too often though, as there is a one-time cost in switching our processes. In terms of impact, we expect that a switch to a longer cycle will encourage more project teams to adopt a "with-intermediary" release model (rather than the traditional "with-rc" single release per cycle), which may lead to abandoning the latter, hence simplifying our processes. Longer cycles might also discourage people to commit to PTL or release liaison work. We'd probably need to manage expectations there, and encourage more frequent switches (or create alternate models). If the decision is made to switch to a longer cycle, the release management team recommends to switch to one year directly. That would avoid changing it again anytime soon, and synchronizing on a calendar year is much simpler to follow and communicate. We also recommend announcing the change well in advance. We currently have an opportunity of making the switch when we reach the end of the release naming alphabet, which would also greatly simplify the communications around the change. Finally, it is worth mentioning the impact on the stable branch work. Releasing less often would likely impact the number of stable branches that we keep on maintaining, so that we do not go too much in the past (and hit unmaintained distributions or long-gone dependencies). We currently maintain releases for 18 months before they switch to extended maintenance, which results in between 3 and 4 releases being maintained at the same time. We'd recommend switching to maintaining one-year releases for 24 months, which would result in between 2 and 3 releases being maintained at the same time. Such a change would lead to longer maintenance for our users while reducing backporting work for our developers. -- Thierry Carrez (ttx) On behalf of the OpenStack Release Management team

15 45

[puppet][tc] Propose changing the release model to cycle-with-rc
by Takashi Kajinami 14 Feb '22

14 Feb '22

Hello, I already discussed this with a few cores last year but I'm raising this topic in ML to make an official decision. Currently Puppet OpenStack is following cycle-with-intermediary and creates a release every milestone. However our code is tightly related to the actual service implementations and having only puppet releases is not very useful. Considering the above point, and effort required to cut off releases per milestone, I'll propose changing our release model to cycle-with-rc , and creating a single release. Because we already created milestone releases for Yoga, I'm thinking of applying the change from next cycle(Z). Please let me know if you have any concerns. Thank you, Takashi

5 5

[Skyline] Project mascot to review
by James Cole 09 Feb '22

09 Feb '22

Greetings everyone, I’m James, a designer with the OpenInfra Foundation. We have a been working on a new project mascot for Skyline and wanted to share a couple of options with you. The reference we were provided alluded to a nine-color-deer, so we created a deer illustration in the style of the other project mascots. The two options are essentially the same, but one is white with spots and the other is yellow with spots. Let me know if you like one or the other—or if we’re totally off on the theme—and we will add the text to the logo and finalize the assets. There is a PDF attached to this message or you can view the document by visiting this link <https://www.dropbox.com/s/d2ikaphzhml7tqb/20220120%20Mascots%20Project.pdf?…>. Thank you! -James

11 20

[kolla] [horizon] Custom logos (WAS: [Openstack-operators] Horizon Custom Logos (Queens, 13.0.1))
by Gregory Orange 09 Feb '22

09 Feb '22

We have been using Ubuntu VMs for the control plane until now, so it was a simple matter of inserting our logo-splash.svg and logo.svg into /var/lib/openstack-dashboard/static/dashboard/img/ and then restarting services. Now we're switching to Kolla, and the relevant path isn't mounted as is the case with the likes of /etc/kolla/horizon and /var/log/kolla. We don't (yet?) build our own container images, so I'm wondering what next. Did anyone get any further with this? On 21/8/18 9:28 pm, Nick Jones wrote: > Hi Erich. > > Yeah, I battled against this myself quite recently. Here's what I did > to add a logo to the Horizon splash page and to the header of each page > itself. > > Create a file called _splash.html, containing: > > <div class="text-center"> > <img class="splash-logo" src={% themable_asset "img/logo.png" %}> > </div> > > And a file called _brand.html, containing: > > {% load branding %} > {% load themes %} > > <a class="navbar-brand" href="{% site_branding_link %}" target="_self"> > <img class="openstack-logo" src="{% themable_asset 'img/logo.png' %}" > alt="{% site_branding %}"> > </a> > > I then created a folder > called /usr/share/openstack-dashboard/openstack_dashboard/themes/default/templates/auth/ > and copied _splash.html into there, copied _brand.html > into /usr/share/openstack-dashboard/openstack_dashboard/templates/header/, > and finally my 'logo.png' was copied > into /usr/lib/python2.7/site-packages/openstack_dashboard/static/dashboard/img/ > > Note that this approach might differ slightly from your setup, as in my > case it's a Kolla-based deployment so these changes are applied to the > image I'm using to deploy a Horizon container. But it's the same > release (Queens) and a CentOS base image, so in principle the steps > should work for you. > > Hope that helps. > > -- > > -Nick > > On 20 August 2018 at 17:40, Erich Weiler <weiler(a)soe.ucsc.edu > <mailto:weiler@soe.ucsc.edu>> wrote: > > Hi Y'all, > > I've been banging my head against a wall for days on this item and > can't find anything via google on how to get around it - I am trying > to install a custom logo onto my Horizon Dashboard front page (the > splash page). I have my logo ready to go, logo-splash.png. I have > tried following the instructions here on how to install a custom logo: > > https://docs.openstack.org/horizon/queens/admin/customize-configure.html > <https://docs.openstack.org/horizon/queens/admin/customize-configure.html> > > But it simply doesn't work. It seems this stanza... > > #splash .login { > background: #355796 url(../img/my_cloud_logo_medium.png) no-repeat > center 35px; > } > > ...doesn't actually replace the logo (which is logo-splash.svg), it > only seems to put my file, logo-splash.png as the *background* to > the .svg logo. And since the option there is "no-repeat center", it > appears *behind* the svg logo and I can't see it. I played around > with those options, removing "no-repeat" for example, and it > dutifully shows my logo repeating in the background. But I need the > default logo-splash.svg file to actually be gone and my logo to > exist in it's place. Maybe I'm missing something simple? > > I'm restarting apache and memchached after every change I make when > I was testing. > > And because the images directory is rebuilt every time I restart > apache, I can't even copy in a custom logo-splash.svg file. Which > wouldn't help anyway, as I want my .png file in there instead. I > don't have the means to create a .svg file at this time. ;) > > Help! > > As a side note, I'm using the Queens distribution via RedHat. > > Many thanks in advance, > erich > > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators(a)lists.openstack.org > <mailto:OpenStack-operators@lists.openstack.org> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators> > > > > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > -- Gregory Orange Cloud System Administrator Scientific Platforms Team building representative Pawsey Supercomputing Centre, CSIRO

3 3

[cinder] new driver freeze & exceptions
by Brian Rosmaita 03 Feb '22

03 Feb '22

Hello Argonauts, The new driver merge deadline passed at 20:00 UTC today. I'm extending the new driver merge deadline to Friday 28 January at 20:00 UTC for two drivers: 1. Lightbits: It has both a cinder and os-brick patch, and I want the team to have more time to look at the os-brick patch. I think the driver patch is close to ready; the developers have been quick to respond to comments and make revisions. Also, the third-party CI is functioning and responding on patches. cinder: https://review.opendev.org/c/openstack/cinder/+/821602 os-brick: https://review.opendev.org/c/openstack/os-brick/+/821603 2. NEC Storage V Series: the driver patch has one +2 and the third party CI is functioning and responding on patches; I don't see any reason to make this one wait for the Z release. https://review.opendev.org/c/openstack/cinder/+/815614 Cinder core reviewers: please continue to review the above patches. With respect to the other proposed drivers: - Pure Storage NVMe-RoCE: vendor decided to hold it for Z - Dell EMC PowerStore NFS driver: vendor decided to hold it for Z - HPE XP Storage FC & iSCSI: the code is very straightforward, but the CI is not ready yet, so this will have to wait for Z - YADRO Tatlin.UNIFIED driver: initial patch arrived yesterday (and has a -1 from Zuul); the CI appears to be running, but the cinder team needs to move on to reviewing os-brick and Yoga feature patches, so this will have to wait for Z also cheers, brian

3 4

[TripleO] Gate blocker - opstools repo
by Ronelle Landy 01 Feb '22

01 Feb '22

Hello All, We have an issue with tripleo-ci-centos-8-content-provider <https://zuul.opendev.org/t/openstack/build/b05d7482835f4b4d83180c29b12973b0> among other tests failing in check/gate due to missing 'centos-opstools' repo. Details are in: https://bugs.launchpad.net/tripleo/+bug/1959619. There are a few patch in flight to try address this issue: https://review.rdoproject.org/r/c/rdo-infra/ansible-role-dlrn/+/38673 - Use opstools packages from trunk.rdoproject.org in cs8 https://review.opendev.org/c/openstack/tripleo-quickstart/+/827111/ - Adds temporary rdoproject repo for collectd/opstools Centos-8 Please hold rechecks until we manage to merge the fixes. Thank you!

2 3

[all][infra][qa][tripleo][openstack-ansible][rally][aodh][cinder][kolla][chef][sahara] CentOS 8 EOL and removal from CI label/image lists
by Clark Boylan 01 Feb '22

01 Feb '22

As noted last month the OpenDev [0] team intends on removing CentOS 8 images from our CI system now that the release has gone EOL. A number of you have already shifted over to CentOS 8 Stream in CI (thank you!), but there is still quite a bit remaining based on codesearch and some manual digging. The OpenDev team has begun the process of removing some of the supporting infrastructure and testing as well [1]. This list is probably not comprehensive but is a start. These projects will need to look at removing their CentOS 8 CI jobs (optionally replacing them with CentOS 8 Stream jobs): * devstack (victoria and older branches) * tripleo-validations * tripleo-upgrade * openstack-ansible * rally * aodh * cinderlib * kolla-ansible * openstack-chef * sahara * tenks * validations-common * validations-lib One thing to keep in mind is that stable branches may also be affected. We'd like to do this cleanup as gracefully as possible, but reality is that some projects are unlikely to completely remove their use of CentOS 8. I think that we can give it until the end of January before we force merge updates to remove the nodeset and label from our configs. At that point any Zuul configuration still using CentOS 8 will enter an error state and correcting it will be necessary to make updates to those Zuul configs. [0] https://lists.opendev.org/pipermail/service-announce/2021-December/000029.h… [1] https://review.opendev.org/q/topic:%22remove-centos-8%22+status:open

7 8

[requirements] The version of zeroconf in upper constraints no longer supports python3.6
by William Szumski 01 Feb '22

01 Feb '22

The zeroconf package seems to have dropped support for python3.6 since 0.38.0, see: https://github.com/jstasiak/python-zeroconf#0380. We currently have 0.38.1 in upper-constraints. My understanding was that the yoga release would still support python3.6. This is important for RHEL8 based distributions that only ship python3.6. Do we need to constrain zeroconf to a version <=0.37.0?

7 6

[infra] Issue with the new pip
by Radosław Piliszek 01 Feb '22

01 Feb '22

Hi! It seems that the new pip is breaking on infra's package indices. Several jobs crash with: """ ValueError: HTML doctype missing or incorrect. Expected <!DOCTYPE html>. If you believe this error to be incorrect, try passing the command line option --use-deprecated=html5lib and please leave a comment on the pip issue at https://github.com/pypa/pip/issues/10825. [0m [31m """ Example from kolla: https://zuul.opendev.org/t/openstack/build/e1bab19dd7c1487e93afeb67284a695e Example from manila: https://zuul.opendev.org/t/openstack/build/984ed9f10f0845ceba59b573e25bbb87 -yoctozepto

4 6