- openstack-discuss - lists.openstack.org

Cinder Bug Report 2023-08-30
by Jon Bernard 30 Aug '23

30 Aug '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - [StorPool Storage] Live VM Migration Fails <https://bugs.launchpad.net/cinder/+bug/2033292> - Status: In Progress - Dell PowerMax - Online migration can fail <https://bugs.launchpad.net/cinder/+bug/2033398> - Status: New - Hitachi: send unnecessary request to unlock storage resource group <https://bugs.launchpad.net/cinder/+bug/2033448> - Status: In Progress Thanks -- Jon

1 0

Re: openstack-discuss Digest, Vol 58, Issue 107
by Karera Tony 30 Aug '23

30 Aug '23

Hi Michael, I have realized that the docker version I have on the new compute nodes after running the bootstrap command is 24 while the previous compute and controller nodes have 20. Could that be a problem? Regards Regards Tony Karera On Wed, Aug 30, 2023 at 3:22 PM < openstack-discuss-request(a)lists.openstack.org> wrote: > Send openstack-discuss mailing list submissions to > openstack-discuss(a)lists.openstack.org > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss > > or, via email, send a message with subject or body 'help' to > openstack-discuss-request(a)lists.openstack.org > > You can reach the person managing the list at > openstack-discuss-owner(a)lists.openstack.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openstack-discuss digest..." > > > Today's Topics: > > 1. Re: [elections][Cinder] PTL Candidacy for 2024.1 (Caracal) > (Brian Rosmaita) > 2. Re: [kolla] [train] [cinder] Cinder issues during controller > replacement (Danny Webb) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 30 Aug 2023 08:31:32 -0400 > From: Brian Rosmaita <rosmaita.fossdev(a)gmail.com> > To: openstack-discuss(a)lists.openstack.org > Subject: Re: [elections][Cinder] PTL Candidacy for 2024.1 (Caracal) > Message-ID: <fa39fe25-6651-688a-4bfa-39abbdbb7e04(a)gmail.com> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On 8/29/23 10:17 AM, Rajat Dhasmana wrote: > > Hi All, > > > > I would like to nominate myself to be Cinder?PTL during the 2024.1 > (Caracal) > > cycle. > [snip] > > Here are some work items we are planning for the next cycle (2024.1): > > > > * We still lack review bandwidth since one of our active cores, Sofia, > > couldn't contribute to Cinder anymore > > hence we will be looking out for potential core reviewers. > > * Continue working on migrating from cinderclient to OSC (and SDK) > support. > > * Continue with the current cinder events like festival of XS reviews, > > midcycle, video meeting once a month etc > > that provides regular team interaction and helps tracking and discussion > > of ongoing work throughout the cycle. > > All this sounds great to me! Thanks for stepping up once again to take > on the task of herding the Argonauts and keeping the Cinder project on > track. > > > > > [1] https://review.opendev.org/q/topic:cinderclient-sdk-migration > > <https://review.opendev.org/q/topic:cinderclient-sdk-migration> > > [2] https://review.opendev.org/q/topic:cinder-sdk-gap > > <https://review.opendev.org/q/topic:cinder-sdk-gap> > > [3] > > > https://docs.google.com/spreadsheets/d/1yetPti2XImRnOXvvJH48yQogdKDzRWDl-NY… > < > https://docs.google.com/spreadsheets/d/1yetPti2XImRnOXvvJH48yQogdKDzRWDl-NY… > > > > [4] > > > https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… > < > https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… > > > > > > Thanks > > Rajat Dhasmana > > > > > ------------------------------ > > Message: 2 > Date: Wed, 30 Aug 2023 13:19:49 +0000 > From: Danny Webb <Danny.Webb(a)thehutgroup.com> > To: Eugen Block <eblock(a)nde.ag>, > "openstack-discuss(a)lists.openstack.org" > <openstack-discuss(a)lists.openstack.org> > Subject: Re: [kolla] [train] [cinder] Cinder issues during controller > replacement > Message-ID: > < > LO2P265MB5773FCAFE86B26D1E45E31459AE6A(a)LO2P265MB5773.GBRP265.PROD.OUTLOOK.COM > > > > Content-Type: text/plain; charset="windows-1252" > > Just to add to the cornucopia of configuration methods for cinder > backends. We use a variation of this for active/active controller setup > which seems to work (on pure, ceph rbd and nimble). We use both > backend_host and volume_backend_name so any of our controllers can action a > request for a given volume. We've not had any issues with this setup since > ussuri (we're on Zed / Yoga now). > > eg: > > [rbd] > backend_host = ceph-nvme > volume_backend_name = high-performance > ... > > [pure] > backend_host = pure > volume_backend_name = high-performance > ... > > which leaves us with a set of the following: > > $ openstack volume service list > > +------------------+-------------------------+----------+---------+-------+----------------------------+ > | Binary | Host | Zone | Status | State > | Updated At | > > +------------------+-------------------------+----------+---------+-------+----------------------------+ > | cinder-volume | nimble@nimble-az1 | gb-lon-1 | enabled | up > | 2023-08-30T12:34:18.000000 | > | cinder-volume | nimble@nimble-az2 | gb-lon-2 | enabled | up > | 2023-08-30T12:34:14.000000 | > | cinder-volume | nimble@nimble-az3 | gb-lon-3 | enabled | up > | 2023-08-30T12:34:17.000000 | > | cinder-volume | ceph-nvme@ceph-nvme-az3 | gb-lon-3 | enabled | up > | 2023-08-30T12:34:14.000000 | > | cinder-volume | pure@pure-az1 | gb-lon-1 | enabled | up > | 2023-08-30T12:34:11.000000 | > | cinder-volume | pure@pure-az2 | gb-lon-2 | enabled | up > | 2023-08-30T12:34:11.000000 | > > +------------------+-------------------------+----------+---------+-------+----------------------------+ > > > ________________________________ > From: Eugen Block <eblock(a)nde.ag> > Sent: 30 August 2023 11:36 > To: openstack-discuss(a)lists.openstack.org < > openstack-discuss(a)lists.openstack.org> > Subject: Re: [kolla] [train] [cinder] Cinder issues during controller > replacement > > CAUTION: This email originates from outside THG > > Just to share our config, we don't use "backend_host" but only "host" > on all control nodes. Its value is the hostname (shortname) pointing > to the virtual IP which migrates in case of a failure. We only use > Ceph as storage backend with different volume types. The volume > service list looks like this: > > controller02:~ # openstack volume service list > > +------------------+--------------------+------+---------+-------+----------------------------+ > | Binary | Host | Zone | Status | State | > Updated At | > > +------------------+--------------------+------+---------+-------+----------------------------+ > | cinder-scheduler | controller | nova | enabled | up | > 2023-08-30T10:32:36.000000 | > | cinder-backup | controller | nova | enabled | up | > 2023-08-30T10:32:34.000000 | > | cinder-volume | controller@rbd | nova | enabled | up | > 2023-08-30T10:32:28.000000 | > | cinder-volume | controller@rbd2 | nova | enabled | up | > 2023-08-30T10:32:36.000000 | > | cinder-volume | controller@ceph-ec | nova | enabled | up | > 2023-08-30T10:32:28.000000 | > > +------------------+--------------------+------+---------+-------+----------------------------+ > > We haven't seen any issue with this setup in years during failover. > > Zitat von Gorka Eguileor <geguileo(a)redhat.com>: > > > On 29/08, Albert Braden wrote: > >> What does backend_host look like? Should it match my internal API > >> URL, i.e. api-int.qde4.ourdomain.com? > >> On Tuesday, August 29, 2023 at 10:44:48 AM EDT, Gorka Eguileor > >> <geguileo(a)redhat.com> wrote: > >> > > > > Hi, > > > > If I remember correctly you can set it to anything you want, but for > > convenience I would recommend setting it to the hostname that currently > > has more volumes in your system. > > > > Let's say you have 3 hosts: > > qde4-ctrl1.cloud.ourdomain.com > > qde4-ctrl2.cloud.ourdomain.com > > qde4-ctrl3.cloud.ourdomain.com > > > > And you have 100 volumes on ctrl1, 20 on ctrl2, and 10 on ctrl3. > > > > Then it would be best to set it to ctrl1: > > [rbd-1] > > backend_host = qde4-ctrl1.cloud.ourdomain.com > > > > And then use the cinder-manage command to modify the other 2. > > > > For your information the value you see as "os-vol-host-attr:host" when > > seeing the detailed information of a volume is in the form of: > > <HostName>@<BackendName>#<PoolName> > > > > In your case: > > <HostName> = qde4-ctrl1.cloud.ourdomain.com > > <BackendName> = rbd-1 > > <PoolName> = rbd-1 > > > > In the RBD case the poolname will always be the same as the backendname. > > > > Cheers, > > Gorka. > > > >> On 29/08, Albert Braden wrote: > >> > We?re replacing controllers, and it takes a few hours to build > >> the new controller. We?re following this procedure to remove the > >> old controller: > >> > https://docs.openstack.org/kolla-ansible/train/user/adding-and-removing-hos… > < > https://docs.openstack.org/kolla-ansible/train/user/adding-and-removing-hos… > > > >> > > >> > After that the cluster seems to run fine on 2 controllers, but > >> approximately 1/3 of our volumes can?t be attached to a VM. When we > >> look at those volumes, we see this: > >> > > >> > | os-vol-host-attr:host | > >> qde4-ctrl1.cloud.ourdomain.com@rbd-1#rbd-1 > >> | > >> > > >> > Ctrl1 is the controller that is being replaced. Is it possible to > >> change the os-vol-host-attr on a volume? How can we work around > >> this issue while we are replacing controllers? Do we need to > >> disable the API for the duration of the replacement process, or is > >> there a better way? > >> > > >> > >> Hi, > >> > >> Assuming you are running cinder volume in Active-Passive mode (which I > >> believe was the only way back in Train) then you should be hardcoding > >> the host name in the cinder.conf file to avoid losing access to your > >> volumes when the volume service starts in another host. > >> > >> This is done with the "backend_host" configuration option within the > >> specific driver section in cinder.conf. > >> > >> As for how to change the value of all the volumes to the same host > >> value, you can use the "cinder-manage" command: > >> > >> cinder-manage volume update_host \ > >> --currenthost <current host> \ > >> --newhost <new host> > >> > >> Cheers, > >> Gorka. > >> > >> > >> > > > >

1 0

[keystone][election] Self-nomination for Keystone PTL for 2024.1 cycle
by Dave Wilde 30 Aug '23

30 Aug '23

Hey folks, It's Dave here, your current OpenStack Keystone PTL. I'd like to submit my candidacy to again act as the PTL for the 2024.1 cycle. We're making great progress with Keystone, and I would like to continue with that excellent work. As PTL in this cycle here are some of the things I'd like to focus on: - Finish the manager role and ensure that the SRBAC implied roles are correct for manager and member - Continue the OAuth 2.0 Implementation - Start a known issues section in the Keystone documentation - Start a documentation audit to ensure that our documentation is of the highest quality Of course we will continue the weekly meetings and the reviewathons. I think the reviewathons have been very successful and I'm keen on keeping them going. I'm looking forward to another successful cycle and working with everyone again! Thank you! /Dave

1 0

[elections][Cinder] PTL Candidacy for 2024.1 (Caracal)
by Rajat Dhasmana 30 Aug '23

30 Aug '23

Hi All, I would like to nominate myself to be Cinder PTL during the 2024.1 (Caracal) cycle. I have been the Cinder PTL for the last 3 cycles namely Zed, 2023.1 (Antelope) and 2023.2 (Bobcat). Few of the notable things we achieved in the past cycle are: ** OpenStack Client and SDK work: *We started two efforts in parallel: ** Migrate OSC commands to SDK[1] ** Add support of missing operations in SDK[2] We are making good progress in this area but looking at the work items[3], this effort will span across the next couple of cycles to be completed. ** Security Vulnerabilities:* We were able to address a major security vulnerability CVE-2023-2088 that prevents end users to delete volume attachments directly leaving left over devices on compute hosts resulting in data leaks. ** Transition from EM to Unmaintained: *We noticed the inactive contribution in extended maintenance branches and the need to fix gate issues from time to time. Also the project team is responsible to merge changes into these EM branches. After the recent CVE-2023-2088 wasn't proposed to EM branches, it was clear that the efforts were not worth the outcome so the Cinder team initiated a discussion regarding EOLing the current EM branches. There is a resolution from TC to migrate EM to unmaintained branches[4]. ** Outreachy Mentorship: *We had a successful quarter of outreachy internship from May-August 2023 with 2 interns, Desire and Toheeb. They were able to get a good amount of work done on the project "Automating generation of api-ref samples". ** Review bandwidth and quality:* We have been able to maintain good quantity and quality of reviews from contributors contributing from different organizations. Here are some work items we are planning for the next cycle (2024.1): * We still lack review bandwidth since one of our active cores, Sofia, couldn't contribute to Cinder anymore hence we will be looking out for potential core reviewers. * Continue working on migrating from cinderclient to OSC (and SDK) support. * Continue with the current cinder events like festival of XS reviews, midcycle, video meeting once a month etc that provides regular team interaction and helps tracking and discussion of ongoing work throughout the cycle. [1] https://review.opendev.org/q/topic:cinderclient-sdk-migration [2] https://review.opendev.org/q/topic:cinder-sdk-gap [3] https://docs.google.com/spreadsheets/d/1yetPti2XImRnOXvvJH48yQogdKDzRWDl-NY… [4] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… Thanks Rajat Dhasmana

2 1

Adding compute nodes to openstack cluster running wallaby
by Karera Tony 30 Aug '23

30 Aug '23

Hello Team, I am running openstack wallaby deployed using kolla-ansible and I am trying to add two compute node but I am getting this error. failed: [compute3] (item={'key': 'cron', 'value': {'container_name': 'cron', 'group': 'cron', 'enabled': True, 'image': 'kolla/ubuntu-source-cron:wallaby', 'environment': {'DUMMY_ENVIRONMENT': 'kolla_useless_env'}, 'volumes': ['/etc/kolla/cron/:/var/lib/kolla/config_files/:ro', '/etc/localtime:/etc/localtime:ro', '/etc/timezone:/etc/timezone:ro', 'kolla_logs:/var/log/kolla/'], 'dimensions': {}}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "cron", "value": {"container_name": "cron", "dimensions": {}, "enabled": true, "environment": {"DUMMY_ENVIRONMENT": "kolla_useless_env"}, "group": "cron", "image": "kolla/ubuntu-source-cron:wallaby", "volumes": ["/etc/kolla/cron/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "/etc/timezone:/etc/timezone:ro", "kolla_logs:/var/log/kolla/"]}}, "msg": "'Traceback (most recent call last):\\n File \"/tmp/ansible_kolla_docker_payload_e49mzp45/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 1241, in main\\n File \"/tmp/ansible_kolla_docker_payload_e49mzp45/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 355, in compare_container\\n File \"/tmp/ansible_kolla_docker_payload_e49mzp45/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 376, in check_container_differs\\n File \"/tmp/ansible_kolla_docker_payload_e49mzp45/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 571, in compare_dimensions\\nKeyError: \\'KernelMemory\\'\\n'"} Please advise Tony Regards Tony Karera

2 1

[release][ironic][OpenStackSDK][swift] cycle-with-intermediary deliverables without any release
by Herve Beraud 30 Aug '23

30 Aug '23

Hi everyone, Quick reminder that we'll need a release very soon for a number of deliverables following a cycle-with-intermediary release model but which have not done *any* release yet in the 2023.2 Bobcat cycle: The following deliverables have done a bobcat release, but it was not refreshed in the last two months: [ironic] bifrost ironic-prometheus-exporter ironic-python-agent-builder ironic-ui networking-baremetal networking-generic-switch [OpenStackSDK] python-openstackclient [swift] swift Those should be released ASAP, and in all cases before September 15, so that we have a release to include in the final 2023.2 Bobcat release. Thanks in advance, -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[horizon] Cancelling Today's Weekly meeting
by vishal manchanda 30 Aug '23

30 Aug '23

Hello Team, As it is a holiday for me, I will not be able to host today's horizon weekly meeting. So let's cancel today's weekly meeting. If anything urgent, please reach out to the horizon core team. Thanks & regards, Vishal Manchanda

1 0

[election][ptl][openstack-helm] PTL Candidacy for 2024.1
by Vladimir Kozhukalov 29 Aug '23

29 Aug '23

Hello Everyone, I am pleased to announce my candidacy to continue as the PTL for OpenStack-Helm in the upcoming 2024.1 release cycle. ## Key accomplishments we have collectively achieved during the Bobcat cycle ### New Features: - Integrated a Manila chart, expanding the range of out-of-the-box use-cases. - Introduced OVN chart and support in Neutron, as OVN is emerging as the de facto standard for OpenStack networking. - Launched a Tacker chart to engage telecom sectors as potential users and contributors. - Upgraded the Elasticsearch chart to version 8.9.0. - Initiated the transition to Rook operator for Ceph management, with completion expected by cycle end. ### Pipeline and Compatibility: - Added Compute-kit and Cinder jobs to the check pipeline for Yoga, Zed, and Antelope releases, thereby extending the range of officially supported Openstack releases. ### Platform Support: - We stay Ubuntu-centric due to lack of resources for support of multiple platforms. During this cycle we introduced Ubuntu Jammy images, and initiated extensive testing for Jammy-based deployments. ### Other Improvements: - Enhanced Compute-kit job stability by adopting a multinode test environment for specific releases. - Cleaned up the OpenStack-Helm storyboard. Over 100 stories were reviewed, verified and fixed or closed. as outdated. - Numerous minor fixes and adjustments, including updates for recent Kubernetes releases. A big thank you to everyone for your contributions. ## Looking Ahead ### Documentation: The documentation still needs to be improved and aligned with all the recent changes. Users need clear understanding of how to deploy/upgrade Openstack components and which releases and scenarios are thoroghly tested and expected to work. ### Compatibility: We plan to continue expanding support for recent OpenStack and Kubernetes releases and additional Linux distributions. I am eager to continue in my role as PTL to ensure these initiatives move forward. Thanks. -- Best regards, Kozhukalov Vladimir

1 0

Nominations closing soon
by Amy Marrich 29 Aug '23

29 Aug '23

Hey all, There is less than 24 hours left for nominations to be committed to the election repos for PTL and TC. If you are a diverse member of the community and see a role you would like to try, especially if you’re a member of a leaderless project, step up and put your name in for a leadership role. There are plenty of us in the community who are willing to help mentor you in these roles. Amy(spotz)

1 0

[election][ptl][tacker] PTL Candidacy for 2024.1
by Yasufumi Ogawa 29 Aug '23

29 Aug '23

Hi, I'd like to nominate myself for Tacker PTL for 2024.1 cycle. I've contributed to Tacker as a PTL to provide a generic VNF Manager to industiries. We have implemented or improved several features such as monitoring with Prometheus for containers or policy management for multi domain usecases in addition to introduce the latest ETSI-NVF standard compliant features. We also revised unstable unit/functional tests for which we have been suffered several times for unexpected failures and waste of time as one of the our goals for efficiency of our development, and documents for users to demonstrate usecases in practical examples. In the next cycle, I'd like to continue to improving Tacker for the 5G network includes both a radio-access network and a core network to provide operators as we've focused on. Deploying Tacker as Service Management and Orchestration (SMO) in O-RAN architecture is one of our plans for realizing the scenario. Tacker team has already contributed to ETSI NFV and O-RAN SC communities while getting feedbacks from them and enforced relationship among them gradually. I'd appreciate if we make some more progress for OpenStack and these communities. Regards, Yasufumi (yasufum)

1 0