April 2022 - openstack-discuss - lists.openstack.org

error
by Firas Hlel 25 Apr '22

25 Apr '22

Firas Hlel <firashlel750(a)gmail.com> jeu. 21 avr. 09:02 (il y a 4 jours) À openstack-discuss Hi all, I am working on a project Openstack in linux ubuntu 20.04. I want to create a cluster hadoop with one master-node and three worker-nodes and i have a problem with a cluster that doesn't work. Status ERROR: Creating cluster failed for the following reason(s): Failed to create trust Error ID: ef5e8b0a-8e6d-4878-bebb-f37f4fa50a88, Failed to create trust Error ID: 43157255-86af-4773-96c1-a07ca7ac66ed. links: https://docs.openstack.org/devstack/latest/ File local.conf : [[local|localrc]] ADMIN_PASSWORD=secret DATABASE_PASSWORD=secret RABBIT_PASSWORD=secret SERVICE_PASSWORD=$ADMIN_PASSWORD HOST_IP=10.0.2.15 LOGFILE=$DEST/logs/stack.sh.log SWIFT_REPLICAS=1SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_DATA_DIR=$DEST/data enable_plugin sahara https://opendev.org/openstack/sahara enable_plugin sahara-dashboard https://opendev.org/openstack/sahara-dashboard enable_plugin heat https://opendev.org/openstack/heat Can you guys advise me about these errors. Is there anything to worry about?

1 0

[openstack-helm] No Meeting This Week
by Gage Hugo 25 Apr '22

25 Apr '22

Hey team, Since I will be busy during our normal meeting time this week and there's nothing on the agenda, the meeting for this week has been cancelled. We will meet again next week at the usual time. Thanks

1 0

[Xena][Cloudkitty] I have no rating in the dashboard
by wodel youchi 24 Apr '22

24 Apr '22

Hi, I've deployed Cloudkitty using Kolla-ansible, then I followed this example to test if the service is working : https://docs.openstack.org/cloudkitty/xena/user/rating/hashmap.html#examples I don't fully understand the service. I waited a day, and until now I don't have any results. The only errors I had is this once is : 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator [-] [scope: a50e58ff494441b3b78ab096bca747d9, worker: 0 ] Error while collecting metric image.size at timestamp 2022-04-01 00:00:00+01:00: {'cause': "Metrics can't being aggregated", 'reason': 'Granularities are missing', 'detail': [['image.size', 'mean', 3600.0]]} (HTTP 400). Exiting.: gnocchiclient.exceptions.BadRequest: {'cause': "Metrics can't being aggregated", 'reason': 'Granularitie:are missing', 'detail': [['image.size', 'mean', 3600.0]]} (HTTP 400) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator Traceback (most recent call last): 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/cloudkitty/orchestrator.py", line 308, in _get_result 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator return self._collect(metric, timestamp) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/cloudkitty/orchestrator.py", line 297, in _collect 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator self._tenant_id, 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/cloudkitty/collector/__init__.py", line 240, in retrieve 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator q_filter=q_filter, 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/cloudkitty/collector/gnocchi.py", line 450, in fetch_all 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator q_filter=q_filter, 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/cloudkitty/collector/gnocchi.py", line 329, in _fetch_metric 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator measurements = self._conn.aggregates.fetch(op, **agg_kwargs) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/gnocchiclient/v1/aggregates.py", line 72, in fetch 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator data=ujson.dumps(data)).json() 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/gnocchiclient/v1/base.py", line 41, in _post 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator return self.client.api.post(*args, **kwargs) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 401, in post 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator return self.request(url, 'POST', **kwargs) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator File "/var/lib/kolla/venv/lib/python3.6/site-packages/gnocchiclient/client.py", line 52, in request 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator raise exceptions.from_response(resp, method) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator gnocchiclient.exceptions.BadRequest: {'cause': "Metrics can't being aggregated", 'reason': 'Granularities are missing', 'detail': [['image.size', 'mean', 3600.0]]} (HTTP 400) 2022-04-20 11:57:37.834 31 ERROR cloudkitty.orchestrator I searched the web and I found this post : https://storyboard.openstack.org/#!/story/2008598 Could that be the problem? If so , how to correct it? i tried to create this file on my kolla-ansible deployer vim /etc/kolla/config/cloudkitty.conf [DEFAULT] debug = True [collect] metrics_conf = /etc/*cloudkitty-api*/metrics.yml (I don't know if this path is correct, since there is no /etc/cloudkitty on the nodes) And I created a directory like this : mkdir /etc/kolla/config/cloudkitty/ wget https://raw.githubusercontent.com/openstack/cloudkitty/master/etc/cloudkitt… -O /etc/kolla/config/cloudkitty/metrics.yml I executed then kolla-ansible -i multinode reconfigure -t cloudkitty All I have in the logs is : 2022-04-21 15:13:32.691 8 INFO cotyledon._service_manager [-] Child 1304 exited with status 1 2022-04-21 15:13:32.693 8 INFO cotyledon._service_manager [-] Child 1310 exited with status 1 2022-04-21 15:13:32.695 8 INFO cotyledon._service_manager [-] Child 1312 exited with status 1 2022-04-21 15:13:32.697 8 INFO cotyledon._service_manager [-] Child 1314 exited with status 1 2022-04-21 15:13:33.097 8 INFO cotyledon._service_manager [-] Child 1325 exited with status 1 2022-04-21 15:13:33.101 8 INFO cotyledon._service_manager [-] Child 1327 exited with status 1 2022-04-21 15:13:33.102 8 INFO cotyledon._service_manager [-] Forking too fast, sleeping 2022-04-21 15:13:38.109 8 INFO cotyledon._service_manager [-] Child 1329 exited with status 1 2022-04-21 15:13:38.112 8 INFO cotyledon._service_manager [-] Child 1331 exited with status 1 2022-04-21 15:13:38.114 8 INFO cotyledon._service_manager [-] Child 1333 exited with status 1 Any ideas? Regards.

1 1

Re: Openstack Trove - Polling request timed out
by Hirotaka Wakabayashi 23 Apr '22

23 Apr '22

Hello Manish! "openstack database instance show" command shows you the current status of an instance. If the status is not ACTIVE, guest-agent fails to start the database service for some reasons. Please see the following page for further debug. https://docs.openstack.org/trove/latest/admin/troubleshooting.html FYI: I think the error occurs here in your case. https://opendev.org/openstack/trove/src/branch/master/trove/taskmanager/mod… Best Regards, Hirotaka On Saturday, April 23, 2022, 12:05:26 AM GMT+9, openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> wrote: Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss or, via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Openstack Trove - Polling request timed out (Manish Bharti) ---------------------------------------------------------------------- Message: 1 Date: Fri, 22 Apr 2022 17:00:31 +0530 From: Manish Bharti <manishbhartigt(a)gmail.com> To: openstack-discuss(a)lists.openstack.org Subject: Openstack Trove - Polling request timed out Message-ID: <CAFTsyOU7D3xf9c4tYb8KhnJQePjCBJvTK_ccHxAOntz_ia69Cg(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Dear Team, We are trying to deploy OpenStack environment for our application and while deploying trove service we are facing the below error(attached screenshot)- Traceback (most recent call last): File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 207, in wait_for_task return polling_task.wait() File "/usr/lib/python3/dist-packages/eventlet/event.py", line 125, in wait result = hub.switch() File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 313, in switch return self.greenlet.switch() File "/usr/lib/python3/dist-packages/oslo_service/loopingcall.py", line 154, in _run_loop idle = idle_for_func(result, self._elapsed(watch)) File "/usr/lib/python3/dist-packages/oslo_service/loopingcall.py", line 349, in _idle_for raise LoopingCallTimeOut( oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 870.99 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/trove/taskmanager/models.py", line 434, in wait_for_instance utils.poll_until(self._service_is_active, File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 223, in poll_until return wait_for_task(task) File "/usr/lib/python3/dist-packages/trove/common/utils.py", line 209, in wait_for_task raise exception.PollTimeOut trove.common.exception.PollTimeOut: Polling request timed out. Please help us on this issue. -- Thank you , Manish Bharti Jodhpur, Raj - 342001. Contact:8875033000

1 0

[all][tc] What's happening in Technical Committee: summary April 15th, 21: Reading: 10 min
by Ghanshyam Mann 22 Apr '22

22 Apr '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on April 21. Most of the meeting discussions are summarized below ( Completed or in-progress activities section). Meeting full logs are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-04-21-15.00.html * Next TC weekly meeting will be on April 28 Thursday at 15:00 UTC, feel free to add the topic on the agenda[1] by April 27. 2. What we completed this week: ========================= * Updated sushy-oem-idrac from x/ to /openstack/ namespace[2] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[3]. We completed the assigned of each item and will continue tracking the progress biweekly. Open Reviews ----------------- * Eight open reviews for ongoing activities[4]. Migration from old ELK service to new Dashboard ----------------------------------------------------------- You might know that Daniel was working to bring the log search dashboard on OpenSearch and he is ready with it and the new dashboard[5] is ready to use. He will send the details about it to ML soon. He is investigating on elactic-recheck instance to be up. The new dashboard is maintained under the OpenStack tact SIG and you can reach out to Daniel on #openstack-infra channel. Please use and provide feedback about this new dashboard to make the upstream failure debugging easy. We also discussed the OpenDev request to shut down the old ELK server and agree to do that. Drop the lower constraints maintenance ------------------------------------------------ The TC resolution to drop the lower constraints testing is under review[6], and open for the feedback[7] Consistent and Secure Default RBAC -------------------------------------------- I have scheduled the discussion for coming Tuesday 26th 14:30 UTC [8], We will target the heat discussion and if time permits then cinder queries. If you have any other questions specific to your project, please add them to the etherpad[9] FIPs community-wide goal ------------------------------- As discussed in PTG, we agreed to select this as per the new milestone. Ade has proposed the milestone update and ready for feedback[10]. Removing the TC Liaisons framework -------------------------------------------- As discussed in PTG, I have proposed the removal of TC liaisons[11] and it's under review. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[12]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's situation again on ML and hope Braden will be ready with their company side permission[13]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[14]. Project updates ------------------- * Add the cinder-three-par charm to Openstack charms[15] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[16]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [17] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/838486 [3] https://etherpad.opendev.org/p/tc-zed-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://docs.openstack.org/project-team-guide/testing.html#checking-status-… [6] https://review.opendev.org/c/openstack/governance/+/838004 [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028199.ht… [8] http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028258.ht… [9] https://etherpad.opendev.org/p/rbac-zed-ptg [10] https://review.opendev.org/c/openstack/governance/+/838601 [11] https://review.opendev.org/c/openstack/governance/+/837891 [12] https://review.opendev.org/c/openstack/governance/+/836888 [13] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [14] https://etherpad.opendev.org/p/zuul-config-error-openstack [15] https://review.opendev.org/c/openstack/governance/+/837781 [16] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [17] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 1

Openstack Ansible error
by Derek O keeffe 22 Apr '22

22 Apr '22

Hi all, I've been working through deploying Openstack ansible when I run the first playbook (setup hosts) it fails with the following error: An exception occurred during task execution. To see the full traceback, use -vvv. The error was: jinja2.exceptions.TemplateRuntimeError: No filter named 'ipaddr' found.failed: [infra1_keystone_container-fb7ae0f1] (item={'key': 'container_address', 'value': {'address': 'xx.xx.xx.xxx', 'bridge': 'br-mgmt', 'interface': 'eth1', 'netmask': '255.255.255.0', 'type': 'veth'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "container_address", "value": {"address": "xx.xx.xx.xxx", "bridge": "br-mgmt", "interface": "eth1", "netmask": "255.255.255.0", "type": "veth"}}, "msg": "TemplateRuntimeError: No filter named 'ipaddr' found."} I have followed this fix: https://bugs.launchpad.net/openstack-ansible/+bug/1963686 But, unfortunately the error still occurs. I am in the process of troubleshooting and if I find a resolution I will post for anyone in a similar position but in the mean time if anyone has a fix/workaround for this issue it would be greatly appreciated. Regards,Derek

2 1

[neutron] Drivers meeting - Friday 22.4.2022 - cancelled
by Lajos Katona 22 Apr '22

22 Apr '22

Hi Neutron Drivers! Due to the lack of agenda, let's cancel today's drivers meeting (sorry for the late mail...). See You on the meeting next week. Lajos Katona (lajoskatona)

1 0

OpenStack Yoga for Ubuntu 22.04 LTS and Ubuntu 20.04 LTS
by Corey Bryant 21 Apr '22

21 Apr '22

The Ubuntu OpenStack team at Canonical is pleased to announce the general availability of OpenStack Yoga on Ubuntu 22.04 LTS (Jammy Jellyfish) and Ubuntu 20.04 LTS (Focal Fossa) via the Ubuntu Cloud Archive. Details of the Yoga release can be found at: https://www.openstack.org/software/yoga To get access to the Ubuntu Yoga packages: == Ubuntu 22.04 LTS == OpenStack Yoga is available by default on Ubuntu 22.04. == Ubuntu 20.04 LTS == The Ubuntu Cloud Archive for OpenStack Yoga can be enabled on Ubuntu 20.04 by running the following command: sudo add-apt-repository cloud-archive:yoga The Ubuntu Cloud Archive for Yoga includes updates for: aodh, barbican, ceilometer, ceph (17.1.0), cinder, designate, designate-dashboard, dpdk (21.11), glance, gnocchi, heat, heat-dashboard, horizon, ironic, ironic-ui, keystone, libvirt (8.0.0), magnum, magnum-ui, manila, manila-ui, masakari, mistral, murano, murano-dashboard, networking-arista, networking-bagpipe, networking-baremetal, networking-bgpvpn, networking-hyperv, networking-l2gw, networking-mlnx, networking-odl, networking-sfc, neutron, neutron-dynamic-routing, neutron-fwaas, neutron-vpnaas, nova, octavia, octavia-dashboard, openstack-trove, openvswitch (2.17.0), ovn (22.03.0), ovn-octavia-provider, placement, sahara, sahara-dashboard, senlin, swift, trove-dashboard, vmware-nsx, vitrage, watcher, watcher-dashboard, zaqar, and zaqar-ui. For a full list of packages and versions, please refer to: https://openstack-ci-reports.ubuntu.com/reports/cloud-archive/yoga_versions… == Reporting bugs == If you have any issues please report bugs using the ‘ubuntu-bug’ tool to ensure that bugs get logged in the right place in Launchpad: sudo ubuntu-bug nova-conductor Thank you to everyone who contributed to OpenStack Yoga! Corey (on behalf of the Ubuntu OpenStack Engineering team)

1 0

[Xena][Gnocchi] gnocchi_statsd become unhealthy after some time
by wodel youchi 21 Apr '22

21 Apr '22

Hi, Recently we deployed Openstack Xena using Kollla-ansible, we noticed that gnocchi_statsd become unhealthy after a period of time, we restarted it several times, it shows a healthy state, but hours after it becomes unhealthy [root@controllera ~]# docker ps | grep gnocch fcc31e240322 192.168.1.16:4000/openstack.kolla/centos-source-gnocchi-statsd:xena "dumb-init --single-…" 45 hours ago Up 24 hours (unhealthy) gnocchi_statsd The log file don't show anything special, you can find all gnocchi logs attached. How can we figure out what causes this?

1 0

error openstack
by Firas Hlel 21 Apr '22

21 Apr '22

Hi all, I am working on a project Openstack in linux ubuntu 20.04. I want to create a cluster hadoop with one master-node and three worker-nodes and i have a problem with a cluster that doesn't work. Status ERROR: Creating cluster failed for the following reason(s): Failed to create trust Error ID: ef5e8b0a-8e6d-4878-bebb-f37f4fa50a88, Failed to create trust Error ID: 43157255-86af-4773-96c1-a07ca7ac66ed. links: https://docs.openstack.org/devstack/latest/ File local.conf : [[local|localrc]] ADMIN_PASSWORD=secret DATABASE_PASSWORD=secret RABBIT_PASSWORD=secret SERVICE_PASSWORD=$ADMIN_PASSWORD HOST_IP=10.0.2.15 LOGFILE=$DEST/logs/stack.sh.log SWIFT_REPLICAS=1SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_DATA_DIR=$DEST/data enable_plugin sahara https://opendev.org/openstack/sahara enable_plugin sahara-dashboard https://opendev.org/openstack/sahara-dashboard enable_plugin heat https://opendev.org/openstack/heat Can you guys advise me about these errors. Is there anything to worry about?

1 0

[glance][devstack][tripleo][ansible][ceph_admin] Glance moving away from single store Configuration
by Abhishek Kekane 21 Apr '22

21 Apr '22

Hello Everyone, Glance has added support to configure multiple stores as a store backend in Stein cycle, and it is very stable now. So in upcoming cycles we are going to remove single store support and use multiple stores support only (PS. you can configure a single store using multiple stores configuration options). As a first step, we have started adding support in devstack [1][2][3] for configuring glance as multiple stores for each of the glance store backend. This cycle we are going to default multistore configuration in devstack so that our gate/check (CI) jobs should test using the same. Following cycles we will start removing single store support from glance code base. If you have any questions related to this work kindly revert back to this mail or you can join us in our weekly meeting, every Thursday at 1400 UTC #openstack-meeting IRC channel as well. [1] https://review.opendev.org/c/openstack/devstack/+/741654 [2] https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/741801/ [3] https://review.opendev.org/c/openstack/devstack/+/741802 Thank you, Abhishek Kekane

3 4

[dev][horizon] Integration of the ec2 feature
by lab reporting 21 Apr '22

21 Apr '22

Hello, We will add EC2 token management within the Horizon project. As a reminder, EC2 Credentials service allows the creation of access/secret credentials used for the ec2 interop layer of OpenStack to authorize S3 requests. It is part of the package keystone.contrib Do you think this development is of interest to the community ? Do you think this development can be built into the core? And if so, should we insert into the panel group dedicated to object storage or into the identity panel ? Thank you in advance for your answers. Best Regards, Tony

1 0

Question on monkey-patching paramiko for FIPS
by Ade Lee 21 Apr '22

21 Apr '22

Hi all, As many have already seen, a number of changes have been merged in OpenStack as part of the effort to allow OpenStack to run on FIPS enabled systems. This effort has been captured in a proposed community goal. [1]. One of the requirements for this effort is that md5sum() not be used in a security related context. In fact, python 3.9 has been modified to raise an exception of hashlib.md5sum() is called on a FIPS enabled system, unless it is explicitly annotated with a usedforsecurity=False attribute [2]. We added a wrapper for md5sum in oslo.config to take advantage of this attribute. [3,4,5] Where we have less control is in libraries used by Openstack - and in particular, paramiko. Paramiko fails on FIPS enabled systems because of a call to md5sum() in get_fingerprint(). A patch has been submitted to fix this problem. [6]. Unfortunately, it takes a very long time for paramiko to fix issues. In order for us to make progress on FIPS testing, a small monkey-patch for paramiko was checked into tempest. [7]. Because this change was made to a test tool, this patch was relatively uncontroversial. A similar change has been found to be needed for manila [8]. I would expect that a similar change will be needed in other components that use paramiko to SSH to other systems (eg. cinder, neutron?) I suspect that the only reason this has not been detected in FIPS testing more widely yet is because the components that use paramiko for SSH are being tested in third party tests that do not, as yet, test FIPS. At the request of the manila team, I am bringing this monkey-patch to the attention of the wider OpenStack community to get feedback on the pros and cons of applying this monkey-patch. A couple things to note: 1. This monkey patch is quite small in scope and only needed until paramiko fixes the issue. 2. paramiko is not FIPS compliant, and so we will ultimately need to fix paramiko or replace it with a different library on FIPS enabled systems. When we do this, we would remove the monkey patch. Thanks, Ade Lee [1] https://opendev.org/openstack/governance/src/branch/master/goals/proposed/f… [2] https://bugs.python.org/issue9216 [3] https://review.opendev.org/c/openstack/oslo.utils/+/750031 [4] Patches to various projects to use oslo.utils adapter for hashlib.md5 (as examples): glance: https://review.opendev.org/c/openstack/glance/+/756158 nova: https://review.opendev.org/c/openstack/nova/+/756434 nova: https://review.opendev.org/c/openstack/nova/+/777686 os-brick: https://review.opendev.org/c/openstack/os-brick/+/756151 oslo: https://review.opendev.org/c/openstack/oslo.versionedobjects/+/756153 tooz: https://review.opendev.org/c/openstack/tooz/+/756432 opensdk: https://review.opendev.org/c/openstack/openstacksdk/+/767411 octavia: https://review.opendev.org/c/openstack/octavia/+/798146 designate: https://review.opendev.org/c/openstack/designate/+/798157 glance_store: https://review.opendev.org/c/openstack/glance_store/+/756157 [5] Swift patch to handle hashlib.md5 https://review.opendev.org/c/openstack/swift/+/751966 [6] https://github.com/paramiko/paramiko/pull/1928 [7] https://review.opendev.org/c/openstack/tempest/+/822560 [8] https://review.opendev.org/c/openstack/manila/+/819375

3 2

Deployment system includes: Ussuri, Brocade X7-8
by Duc Loi 21 Apr '22

21 Apr '22

Hi Everyone, In the near future, we will deploy a system including Ussuri and Brocade X7-8 according to customer requirements. But Brocade has made a notice not to support versions after Train. Has anyone implemented a system similar? What are the challenges you face? What solutions can fix it? Thanks and Best Regards

1 0

[all][tc] Technical Committee next weekly meeting on April 21 at 1500 UTC
by Ghanshyam Mann 21 Apr '22

21 Apr '22

Hello Everyone, Technical Committee's next weekly meeting is scheduled for April 21 at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, April 20, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

OPS Meetup Registration
by Amy Marrich 20 Apr '22

20 Apr '22

Registration is now open for the OPS Meetup to be held Friday right after the Summit. https://www.eventbrite.com/e/openstack-ops-meetup-tickets-322813472787 And don't forget you can still add topics: https://etherpad.opendev.org/p/april2022-ptg-openstack-ops Thanks, Amy (spotz)

1 0

[cinder] Bug deputy report for week of 04-20-2022
by Sofia Enriquez 20 Apr '22

20 Apr '22

This is a bug report from 04-13-2022 to 04-20-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/cinder/+bug/1969366 "Backend incorrectly reporting cacheable capability." Assigned to Gorka. - https://bugs.launchpad.net/cinder/+bug/1969408 "[RBD] Cinder fails to retype / migrate large volumes." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1967481 "[Storwize_SVC]Retype operation failure for GMCV due to update_clean_rate." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1968164 "Incorrectly shown volume size while using PowerFlex cinder driver." Fix proposed to master. Low - https://bugs.launchpad.net/cinder/+bug/1967686 "[DEFAULT] use_forwarded_for is a duplicate of the HTTPProxyToWSGI middleware." Assigned to Takashi Kajinami. - https://bugs.launchpad.net/cinder/+bug/1968048 "Pylint tox environment fails on Fedora 35." Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1968159 "[IBM Storwize] Retype failure for replication volume-type." Fix proposed to master. Invalid - https://bugs.launchpad.net/cinder/+bug/1969213 "When image properties contain "signature_verified" field，create volume from image wil fail." Unassigned. Cheers -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

ERROR CREATING CLUSTER
by Firas Dehech 20 Apr '22

20 Apr '22

Hi all, I am working on a project Openstack in linux ubuntu 20.04. I want to create a cluster hadoop with one master-node and three worker-nodes and i have a problem with a cluster that doesn't work. Status ERROR: Creating cluster failed for the following reason(s): Failed to create trust Error ID: ef5e8b0a-8e6d-4878-bebb-f37f4fa50a88, Failed to create trust Error ID: 43157255-86af-4773-96c1-a07ca7ac66ed. links: https://docs.openstack.org/devstack/latest/ File local.conf : [[local|localrc]] ADMIN_PASSWORD=secret DATABASE_PASSWORD=secret RABBIT_PASSWORD=secret SERVICE_PASSWORD=$ADMIN_PASSWORD HOST_IP=10.0.2.15 LOGFILE=$DEST/logs/stack.sh.log SWIFT_REPLICAS=1SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_DATA_DIR=$DEST/data enable_plugin sahara https://opendev.org/openstack/sahara enable_plugin sahara-dashboard https://opendev.org/openstack/sahara-dashboard enable_plugin heat https://opendev.org/openstack/heat Can you guys advise me about these errors. Is there anything to worry about? Regards.

1 0

[security][security sig] Stepping down as chair
by Gage Hugo 19 Apr '22

19 Apr '22

Hey folks, I've been serving as the Security SIG chair for over 4 years now and throughout my entire time as chair I've had the wonderful experience of learning and working alongside several excellent individuals. It also has given me the great opportunity to have multiple discussions on the topic of security with the community as a whole which helped me grow my knowledge base and career. However, with my focus shifting away from pure OpenStack work and the rather long time that 4 years is, I believe it's time for someone with a fresh perspective to have the opportunity to step into the role and help guide the Security SIG into the future. I am stepping down from the role of Security SIG chair as of today, but I plan on sticking around in the SIG for as long as I can. I hope to continue contributing when I can and I will be around to help anyone who is interested in the role of chair. I'd like to give a big thank you to Jeremy Stanley (fungi) for all the help he provided throughout my time as chair and continues to do so to this day whenever I have a question and need help. He helped me step into the chair role and I am sure he will also help anyone who is interested in the role as well. Thanks again for the opportunity and I hope to see you all around! - Gage Hugo

1 0

[Kolla-ansible][Xena][Ceph-RGW] need help configuring Ceph RGW for Swift and S3 access
by wodel youchi 19 Apr '22

19 Apr '22

Hi, I am having trouble configuring Openstack to use Ceph RGW as the Object store backend for Swift and S3. My setup is an HCI, I have 3 controllers which are also my ceph mgrs, mons and rgws and 9 compte/storage servers (osds). Xena is deployed with Ceph Pacific. Ceph public network is a private network on vlan10 with 10.10.1.0/24 as a subnet. Here is a snippet from my globals.yml : > --- > kolla_base_distro: "centos" > kolla_install_type: "source" > openstack_release: "xena" > kolla_internal_vip_address: "10.10.3.1" > kolla_internal_fqdn: "dashint.cloud.example.com" > kolla_external_vip_address: "x.x.x.x" > kolla_external_fqdn: "dash.cloud.example.com " > docker_registry: 192.168.1.16:4000 > network_interface: "bond0" > kolla_external_vip_interface: "bond1" > api_interface: "bond1.30" > *storage_interface: "bond1.10" <---------------- VLAN10 (public ceph > network)* > tunnel_interface: "bond1.40" > dns_interface: "bond1" > octavia_network_interface: "bond1.301" > neutron_external_interface: "bond2" > neutron_plugin_agent: "openvswitch" > keepalived_virtual_router_id: "51" > kolla_enable_tls_internal: "yes" > kolla_enable_tls_external: "yes" > kolla_certificates_dir: "{{ node_config }}/certificates" > kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" > kolla_internal_fqdn_cert: "{{ kolla_certificates_dir > }}/haproxy-internal.pem" > kolla_admin_openrc_cacert: "{{ kolla_certificates_dir }}/ca.pem" > kolla_copy_ca_into_containers: "yes" > kolla_enable_tls_backend: "yes" > kolla_verify_tls_backend: "no" > kolla_tls_backend_cert: "{{ kolla_certificates_dir }}/backend-cert.pem" > kolla_tls_backend_key: "{{ kolla_certificates_dir }}/backend-key.pem" > enable_openstack_core: "yes" > enable_hacluster: "yes" > enable_haproxy: "yes" > enable_aodh: "yes" > enable_barbican: "yes" > enable_ceilometer: "yes" > enable_central_logging: "yes" > > *enable_ceph_rgw: "yes"enable_ceph_rgw_loadbalancer: "{{ enable_ceph_rgw | > bool }}"* > enable_cinder: "yes" > enable_cinder_backup: "yes" > enable_collectd: "yes" > enable_designate: "yes" > enable_elasticsearch_curator: "yes" > enable_freezer: "no" > enable_gnocchi: "yes" > enable_gnocchi_statsd: "yes" > enable_magnum: "yes" > enable_manila: "yes" > enable_manila_backend_cephfs_native: "yes" > enable_mariabackup: "yes" > enable_masakari: "yes" > enable_neutron_vpnaas: "yes" > enable_neutron_qos: "yes" > enable_neutron_agent_ha: "yes" > enable_neutron_provider_networks: "yes" > enable_neutron_segments: "yes" > enable_octavia: "yes" > enable_trove: "yes" > external_ceph_cephx_enabled: "yes" > ceph_glance_keyring: "ceph.client.glance.keyring" > ceph_glance_user: "glance" > ceph_glance_pool_name: "images" > ceph_cinder_keyring: "ceph.client.cinder.keyring" > ceph_cinder_user: "cinder" > ceph_cinder_pool_name: "volumes" > ceph_cinder_backup_keyring: "ceph.client.cinder-backup.keyring" > ceph_cinder_backup_user: "cinder-backup" > ceph_cinder_backup_pool_name: "backups" > ceph_nova_keyring: "{{ ceph_cinder_keyring }}" > ceph_nova_user: "cinder" > ceph_nova_pool_name: "vms" > ceph_gnocchi_keyring: "ceph.client.gnocchi.keyring" > ceph_gnocchi_user: "gnocchi" > ceph_gnocchi_pool_name: "metrics" > ceph_manila_keyring: "ceph.client.manila.keyring" > ceph_manila_user: "manila" > glance_backend_ceph: "yes" > glance_backend_file: "no" > gnocchi_backend_storage: "ceph" > cinder_backend_ceph: "yes" > cinder_backup_driver: "ceph" > cloudkitty_collector_backend: "gnocchi" > designate_ns_record: "cloud.example.com " > nova_backend_ceph: "yes" > nova_compute_virt_type: "kvm" > octavia_auto_configure: yes > octavia_amp_flavor: > name: "amphora" > is_public: no > vcpus: 1 > ram: 1024 > disk: 5 > octavia_amp_network: > name: lb-mgmt-net > provider_network_type: vlan > provider_segmentation_id: 301 > provider_physical_network: physnet1 > external: false > shared: false > subnet: > name: lb-mgmt-subnet > cidr: "10.7.0.0/16" > allocation_pool_start: "10.7.0.50" > allocation_pool_end: "10.7.255.200" > no_gateway_ip: yes > enable_dhcp: yes > mtu: 9000 > octavia_amp_network_cidr: 10.10.7.0/24 > octavia_amp_image_tag: "amphora" > octavia_certs_country: XZ > octavia_certs_state: Gotham > octavia_certs_organization: WAYNE > octavia_certs_organizational_unit: IT > horizon_keystone_multidomain: true > elasticsearch_curator_dry_run: "no" > enable_cluster_user_trust: true > > > > > > > > > > > > *ceph_rgw_hosts: - host: controllera ip: 10.10.1.5 > port: 8080 - host: controllerb ip: 10.10.1.9 > port: 8080 - host: controllerc ip: 10.10.1.13 > port: 8080ceph_rgw_swift_account_in_url: trueceph_rgw_swift_compatibility: > true* And Here is my ceph all.yml file > --- > dummy: > ceph_release_num: 16 > cluster: ceph > configure_firewall: False > *monitor_interface: bond1.10* > monitor_address_block: 10.10.1.0/24 > is_hci: true > hci_safety_factor: 0.2 > osd_memory_target: 4294967296 > *public_network: 10.10.1.0/24 <http://10.10.1.0/24>* > cluster_network: 10.10.2.0/24 > *radosgw_interface: "{{ monitor_interface }}"* > *radosgw_address_block: 10.10.1.0/24 <http://10.10.1.0/24>* > nfs_file_gw: true > nfs_obj_gw: true > ceph_docker_image: "ceph/daemon" > ceph_docker_image_tag: latest-pacific > ceph_docker_registry: 192.168.1.16:4000 > containerized_deployment: True > openstack_config: true > openstack_glance_pool: > name: "images" > pg_autoscale_mode: False > application: "rbd" > pg_num: 128 > pgp_num: 128 > target_size_ratio: 5.00 > rule_name: "SSD" > openstack_cinder_pool: > name: "volumes" > pg_autoscale_mode: False > application: "rbd" > pg_num: 1024 > pgp_num: 1024 > target_size_ratio: 42.80 > rule_name: "SSD" > openstack_nova_pool: > name: "vms" > pg_autoscale_mode: False > application: "rbd" > pg_num: 256 > pgp_num: 256 > target_size_ratio: 10.00 > rule_name: "SSD" > openstack_cinder_backup_pool: > name: "backups" > pg_autoscale_mode: False > application: "rbd" > pg_num: 512 > pgp_num: 512 > target_size_ratio: 18.00 > rule_name: "SSD" > openstack_gnocchi_pool: > name: "metrics" > pg_autoscale_mode: False > application: "rbd" > pg_num: 32 > pgp_num: 32 > target_size_ratio: 0.10 > rule_name: "SSD" > openstack_cephfs_data_pool: > name: "cephfs_data" > pg_autoscale_mode: False > application: "cephfs" > pg_num: 256 > pgp_num: 256 > target_size_ratio: 10.00 > rule_name: "SSD" > openstack_cephfs_metadata_pool: > name: "cephfs_metadata" > pg_autoscale_mode: False > application: "cephfs" > pg_num: 32 > pgp_num: 32 > target_size_ratio: 0.10 > rule_name: "SSD" > openstack_pools: > - "{{ openstack_glance_pool }}" > - "{{ openstack_cinder_pool }}" > - "{{ openstack_nova_pool }}" > - "{{ openstack_cinder_backup_pool }}" > - "{{ openstack_gnocchi_pool }}" > - "{{ openstack_cephfs_data_pool }}" > - "{{ openstack_cephfs_metadata_pool }}" > openstack_keys: > - { name: client.glance, caps: { mon: "profile rbd", osd: "profile rbd > pool={{ openstack_cinder_pool.name }}, profile rbd pool={{ > openstack_glance_pool.name }}"}, mode: "0600" } > - { name: client.cinder, caps: { mon: "profile rbd", osd: "profile rbd > pool={{ openstack_cinder_pool.name }}, profile rbd pool={{ > openstack_nova_pool.name }}, profile rbd pool={{ > openstack_glance_pool.name }}"}, mode: "0600" } > - { name: client.cinder-backup, caps: { mon: "profile rbd", osd: > "profile rbd pool={{ openstack_cinder_backup_pool.name }}"}, mode: "0600" > } > - { name: client.gnocchi, caps: { mon: "profile rbd", osd: "profile rbd > pool={{ openstack_gnocchi_pool.name }}"}, mode: "0600", } > - { name: client.openstack, caps: { mon: "profile rbd", osd: "profile > rbd pool={{ openstack_glance_pool.name }}, profile rbd pool={{ > openstack_nova_pool.name }}, profile rbd pool={{ > openstack_cinder_pool.name }}, profile rbd pool={{ > openstack_cinder_backup_pool.name }}"}, mode: "0600" } > dashboard_enabled: True > dashboard_protocol: https > dashboard_port: 8443 > dashboard_network: "192.168.1.0/24" > dashboard_admin_user: admin > dashboard_admin_user_ro: true > dashboard_admin_password: *********** > dashboard_crt: '/home/deployer/work/site-central/chaininv.crt' > dashboard_key: '/home/deployer/work/site-central/cloud_example.com.priv' > dashboard_grafana_api_no_ssl_verify: true > dashboard_rgw_api_user_id: admin > dashboard_rgw_api_no_ssl_verify: true > dashboard_frontend_vip: '192.168.1.5' > node_exporter_container_image: " > 192.168.1.16:4000/prom/node-exporter:v0.17.0" > grafana_admin_user: admin > grafana_admin_password: ********* > grafana_crt: '/home/deployer/work/site-central/chaininv.crt' > grafana_key: '/home/deployer/work/site-central/cloud_example.com.priv' > grafana_server_fqdn: 'grafanasrv.cloud.example.com' > grafana_container_image: "192.168.1.16:4000/grafana/grafana:6.7.4" > grafana_dashboard_version: pacific > prometheus_container_image: "192.168.1.16:4000/prom/prometheus:v2.7.2" > alertmanager_container_image: "192.168.1.16:4000/prom/alertmanager:v0.16.2 > " > And my rgws.yml > --- > dummy: > copy_admin_key: true > rgw_create_pools: > "{{ rgw_zone }}.rgw.buckets.data": > pg_num: 256 > pgp_num: 256 > size: 3 > type: replicated > pg_autoscale_mode: False > rule_id: 1 > "{{ rgw_zone }}.rgw.buckets.index": > pg_num: 64 > pgp_num: 64 > size: 3 > type: replicated > pg_autoscale_mode: False > rule_id: 1 > "{{ rgw_zone }}.rgw.meta": > pg_num: 32 > pgp_num: 32 > size: 3 > type: replicated > pg_autoscale_mode: False > rule_id: 1 > "{{ rgw_zone }}.rgw.log": > pg_num: 32 > pgp_num: 32 > size: 3 > type: replicated > pg_autoscale_mode: False > rule_id: 1 > "{{ rgw_zone }}.rgw.control": > pg_num: 32 > pgp_num: 32 > size: 3 > type: replicated > pg_autoscale_mode: False > rule_id: 1 > The ceph_rgw user was created by kolla (xenavenv) [deployer@rscdeployer ~]$ openstack user list | grep ceph | 3262aa7e03ab49c8a5710dfe3b16a136 | ceph_rgw This is my ceph.conf from one of my controllers : > [root@controllera ~]# cat /etc/ceph/ceph.conf > [client.rgw.controllera.rgw0] > host = controllera > rgw_keystone_url = https://dash.cloud.example.com:5000 > ##Authentication using username, password and tenant. Preferred. > rgw_keystone_verify_ssl = false > rgw_keystone_api_version = 3 > rgw_keystone_admin_user = ceph_rgw > rgw_keystone_admin_password = cos2Jcnpnw9BhGwvPm************************** > rgw_keystone_admin_domain = Default > rgw_keystone_admin_project = service > rgw_s3_auth_use_keystone = true > rgw_keystone_accepted_roles = admin > rgw_keystone_implicit_tenants = true > rgw_swift_account_in_url = true > keyring = /var/lib/ceph/radosgw/ceph-rgw.controllera.rgw0/keyring > log file = /var/log/ceph/ceph-rgw-controllera.rgw0.log > rgw frontends = beast endpoint=10.10.1.5:8080 > rgw thread pool size = 512 > #For Debug > debug ms = 1 > debug rgw = 20 > > > # Please do not change this file directly since it is managed by Ansible > and will be overwritten > [global] > cluster network = 10.10.2.0/24 > fsid = da094354-6ade-415a-a424-************ > mon host = [v2:10.10.1.5:3300,v1:10.10.1.5:6789],[v2:10.10.1.9:3300,v1: > 10.10.1.9:6789],[v2:10.10.1.13:3300,v1:10.10.1.13:6789] > mon initial members = controllera,controllerb,controllerc > osd pool default crush rule = 1 > *public network = 10.10.1.0/24 <http://10.10.1.0/24>* > Here are my swift endpoints (xenavenv) [deployer@rscdeployer ~]$ openstack endpoint list | grep swift | 4082b4acf8bc4e4c9efc6e2d0e293724 | RegionOne | swift | object-store | True | admin | https://dashint.cloud.example.com:6780/v1/AUTH_%(project_id)s | | b13a2f53e13e4650b4efdb8184eb0211 | RegionOne | swift | object-store | True | internal | https://dashint.cloud.example.com:6780/v1/AUTH_%(project_id)s | | f85b36ff9a2b49bc9eaadf1aafdee28c | RegionOne | swift | object-store | True | public | https://dash.cloud.example.com:6780/v1/AUTH_%(project_id)s | When I connect to Horizon -> Project -> Object Store -> Containers I get theses errors : - Unable to get the swift container listing - Unable to fetch the policy details. I cannot create a new container from the WebUI, the Storage policy parameter is empty. If I try to create a new container from the CLI, I get this : (xenavenv) [deployer@rscdeployer ~]$ source cephrgw-openrc.sh (xenavenv) [deployer@rscdeployer ~]$ openstack container create demo -v START with options: container create demo -v command: container create -> openstackclient.object.v1.container.CreateContainer (auth=True) Using auth plugin: password Not Found (HTTP 404) END return value: 1 This is the log from RGW service when I execute the above command : > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 CONTENT_LENGTH=0 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 HTTP_ACCEPT=*/* > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 HTTP_ACCEPT_ENCODING=gzip, > deflate > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 HTTP_HOST= > dashint.cloud.example.com:6780 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 > HTTP_USER_AGENT=openstacksdk/0.59.0 keystoneauth1/4.4.0 > python-requests/2.26.0 CPython/3.8.8 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 HTTP_VERSION=1.1 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 > HTTP_X_AUTH_TOKEN=gAAAAABiXUrjDFNzXx03mt1lbpUiCqNND1HACspSfg6h_TMxKYND5Hb9BO3FxH0a7CYoBXgRJywGszlK8cl-7zbUNRjHmxgIzmyh-CrWyGv793ZLOAmT_XShcrIKThjIIH3gTxYoX1TXwOKbsvMuZnI5EKKsol2y2MhcqPLeLGc28_AwoOr_b80 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 > HTTP_X_FORWARDED_FOR=10.10.3.16 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 HTTP_X_FORWARDED_PROTO=https > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 REMOTE_ADDR=10.10.1.13 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 REQUEST_METHOD=PUT > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 > REQUEST_URI=/v1/AUTH_971efa4cb18f42f7a405342072c39c9d/demo > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 > SCRIPT_URI=/v1/AUTH_971efa4cb18f42f7a405342072c39c9d/demo > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 SERVER_PORT=8080 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 1 ====== starting new request > req=0x7f23221aa620 ===== > 2022-04-18T12:26:27.995+0100 7f22e07a9700 2 req 728157015944164764 > 0.000000000s initializing for trans_id = > tx000000a1aeef2b40f759c-00625d4ae3-4b389-default > 2022-04-18T12:26:27.995+0100 7f22e07a9700 10 req 728157015944164764 > 0.000000000s rgw api priority: s3=8 s3website=7 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 10 req 728157015944164764 > 0.000000000s host=dashint.cloud.example.com > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 req 728157015944164764 > 0.000000000s subdomain= domain= in_hosted_domain=0 > in_hosted_domain_s3website=0 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 req 728157015944164764 > 0.000000000s final domain/bucket subdomain= domain= in_hosted_domain=0 > in_hosted_domain_s3website=0 s->info.domain= > s->info.request_uri=/v1/AUTH_971efa4cb18f42f7a405342072c39c9d/demo > 2022-04-18T12:26:27.995+0100 7f22e07a9700 20 req 728157015944164764 > 0.000000000s get_handler handler=22RGWHandler_REST_Obj_S3 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 10 req 728157015944164764 > 0.000000000s handler=22RGWHandler_REST_Obj_S3 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 2 req 728157015944164764 > 0.000000000s getting op 1 > 2022-04-18T12:26:27.995+0100 7f22e07a9700 1 -- 10.10.1.13:0/2715436964 > --> [v2:10.10.1.7:6801/4815,v1:10.10.1.7:6803/4815] -- > osd_op(unknown.0.0:1516 12.3 12:c14cb721:::script.prerequest.:head [call > version.read in=11b,getxattrs,stat] snapc 0=[] > ondisk+read+known_if_redirected e1182) v8 -- 0x56055eb2c400 con > 0x56055e53b000 > 2022-04-18T12:26:27.996+0100 7f230d002700 1 -- 10.10.1.13:0/2715436964 > <== osd.23 v2:10.10.1.7:6801/4815 22 ==== osd_op_reply(1516 > script.prerequest. [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such > file or directory)) v8 ==== 246+0+0 (crc 0 0 0) 0x56055ea18b40 con > 0x56055e53b000 > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 10 req 728157015944164764 > 0.001000002s s3:put_obj scheduling with throttler client=2 cost=1 > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 10 req 728157015944164764 > 0.001000002s s3:put_obj op=21RGWPutObj_ObjStore_S3 > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 2 req 728157015944164764 > 0.001000002s s3:put_obj verifying requester > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 20 req 728157015944164764 > 0.001000002s s3:put_obj rgw::auth::StrategyRegistry::s3_main_strategy_t: > trying rgw::auth::s3::AWSAuthStrategy > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 20 req 728157015944164764 > 0.001000002s s3:put_obj rgw::auth::s3::AWSAuthStrategy: trying > rgw::auth::s3::S3AnonymousEngine > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 20 req 728157015944164764 > 0.001000002s s3:put_obj rgw::auth::s3::S3AnonymousEngine granted access > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 20 req 728157015944164764 > 0.001000002s s3:put_obj rgw::auth::s3::AWSAuthStrategy granted access > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 2 req 728157015944164764 > 0.001000002s s3:put_obj normalizing buckets and tenants > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 10 req 728157015944164764 > 0.001000002s s->object=AUTH_971efa4cb18f42f7a405342072c39c9d/demo > s->bucket=v1 > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 2 req 728157015944164764 > 0.001000002s s3:put_obj init permissions > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 20 req 728157015944164764 > 0.001000002s s3:put_obj get_system_obj_state: rctx=0x7f23221a9000 > obj=default.rgw.meta:root:v1 state=0x56055ea8c520 s->prefetch_data=0 > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 10 req 728157015944164764 > 0.001000002s s3:put_obj cache get: name=default.rgw.meta+root+v1 : miss > 2022-04-18T12:26:27.996+0100 7f22ddfa4700 1 -- 10.10.1.13:0/2715436964 > --> [v2:10.10.1.3:6802/4933,v1:10.10.1.3:6806/4933] -- > osd_op(unknown.0.0:1517 11.b 11:d05f7b30:root::v1:head [call version.read > in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e1182) v8 > -- 0x56055eb2cc00 con 0x56055e585000 > 2022-04-18T12:26:27.997+0100 7f230c801700 1 -- 10.10.1.13:0/2715436964 > <== osd.3 v2:10.10.1.3:6802/4933 9 ==== osd_op_reply(1517 v1 > [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) > v8 ==== 230+0+0 (crc 0 0 0) 0x56055e39db00 con 0x56055e585000 > 2022-04-18T12:26:27.997+0100 7f22dd7a3700 10 req 728157015944164764 > 0.002000004s s3:put_obj cache put: name=default.rgw.meta+root+v1 > info.flags=0x0 > 2022-04-18T12:26:27.997+0100 7f22dd7a3700 10 req 728157015944164764 > 0.002000004s s3:put_obj adding default.rgw.meta+root+v1 to cache LRU end > 2022-04-18T12:26:27.997+0100 7f22dd7a3700 10 req 728157015944164764 > 0.002000004s s3:put_obj init_permissions on <NULL> failed, ret=-2002 > 2022-04-18T12:26:27.997+0100 7f22dd7a3700 1 req 728157015944164764 > 0.002000004s op->ERRORHANDLER: err_no=-2002 new_err_no=-2002 > 2022-04-18T12:26:27.997+0100 7f22dbfa0700 1 -- 10.10.1.13:0/2715436964 > --> [v2:10.10.1.8:6804/4817,v1:10.10.1.8:6805/4817] -- > osd_op(unknown.0.0:1518 12.1f 12:fb11263f:::script.postrequest.:head [call > version.read in=11b,getxattrs,stat] snapc 0=[] > ondisk+read+known_if_redirected e1182) v8 -- 0x56055eb2d000 con > 0x56055e94c800 > 2022-04-18T12:26:27.998+0100 7f230d002700 1 -- 10.10.1.13:0/2715436964 > <== osd.9 v2:10.10.1.8:6804/4817 10 ==== osd_op_reply(1518 > script.postrequest. [call,getxattrs,stat] v0'0 uv0 ondisk = -2 ((2) No such > file or directory)) v8 ==== 247+0+0 (crc 0 0 0) 0x56055ea18b40 con > 0x56055e94c800 > 2022-04-18T12:26:27.998+0100 7f22d8f9a700 2 req 728157015944164764 > 0.003000006s s3:put_obj op status=0 > 2022-04-18T12:26:27.998+0100 7f22d8f9a700 2 req 728157015944164764 > 0.003000006s s3:put_obj http status=404 > 2022-04-18T12:26:27.998+0100 7f22d8f9a700 1 ====== req done > req=0x7f23221aa620 op status=0 http_status=404 latency=0.003000006s ====== > 2022-04-18T12:26:27.998+0100 7f22d8f9a700 1 beast: 0x7f23221aa620: > 10.10.1.13 - anonymous [18/Apr/2022:12:26:27.995 +0100] "PUT > /v1/AUTH_971efa4cb18f42f7a405342072c39c9d/demo HTTP/1.1" 404 214 - > "openstacksdk/0.59.0 keystoneauth1/4.4.0 python-requests/2.26.0 > CPython/3.8.8" - latency=0.003000006s > Could you help please. Regards.

2 4

[Kolla][Kolla-Ansible] Unable to Deploy tacker victoria in centos 8 Stream
by Richa Gupta 19 Apr '22

19 Apr '22

We tried installing the tacker victoria release using Kolla Ansible all-in-one deployment on centos stream 8, by using the following links: https://docs.openstack.org/tacker/victoria/install/kolla.html and running prerequisites mentioned in the link below: https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html#instal… (We changed ussri to victoria at all necessary places. The globals.yml exactly matches with the one given in the OpenStack link) while running deployment we encounter the following error: RUNNING HANDLER [Waiting for rabbitmq to start] ********************************************************************************************************** fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["docker", "exec", "rabbitmq", "rabbitmqctl", "wait", "/var/lib/rabbitmq/mnesia/rabbitmq.pid"], "delta": "0:00:10.704454", "end": "2022-04-19 11:43:12.259809", "msg": "non-zero return code", "rc": 75, "start": "2022-04-19 11:43:01.555355", "stderr": "Error: operation wait on node rabbit@tacker-victoria timed out. Timeout value used: 10000", "stderr_lines": ["Error: operation wait on node rabbit@tacker-victoria timed out. Timeout value used: 10000"], "stdout": "Waiting for pid file '/var/lib/rabbitmq/mnesia/rabbitmq.pid' to appear", "stdout_lines": ["Waiting for pid file '/var/lib/rabbitmq/mnesia/rabbitmq.pid' to appear"]} RUNNING HANDLER [Restart remaining rabbitmq containers] ************************************************************************************************** NO MORE HOSTS LEFT *************************************************************************************************************************************** PLAY RECAP *********************************************************************************************************************************************** localhost : ok=77 changed=43 unreachable=0 failed=1 skipped=19 rescued=0 ignored=1 After this we run deploy again and encounter the following error: TASK [service-rabbitmq : nova | Ensure RabbitMQ users exist] *************************************************************************************************************************** FAILED - RETRYING: nova | Ensure RabbitMQ users exist (5 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (4 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (3 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (2 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (1 retries left). failed: [localhost -> localhost] (item=None) => {"attempts": 5, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} fatal: [localhost -> {{ service_rabbitmq_delegate_host }}]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} PLAY RECAP ***************************************************************************************************************************************************************************** localhost : ok=116 changed=42 unreachable=0 failed=1 skipped=40 rescued=0 ignored=0 Command failed ansible-playbook -i /usr/local/share/kolla-ansible/ansible/inventory/all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=deploy /usr/local/share/kolla-ansible/ansible/site.yml [centos@tacker-victoria ~]$

1 0

[murano][octavia][sahara][zaqar][zun][oslo] Pending removal of 'oslo_db.sqlalchemy.test_base'
by Stephen Finucane 19 Apr '22

19 Apr '22

o/ This is a heads up to the maintainers of the aforementioned projects that the oslo team are planning to remove the 'oslo_db.sqlalchemy.test_base' module this cycle. This module has been deprecated since 2015 and we want to get rid of it to reduce load on the overburdened oslo maintainers. I have already fixed the issue in a couple of projects. These can be used as blueprints for fixing the remaining affected projects: * masakari (https://review.opendev.org/c/openstack/masakari/+/802761) * glance (https://review.opendev.org/c/openstack/glance/+/802762) * manila (https://review.opendev.org/c/openstack/manila/+/802763) I would love to fix the remaining projects but my limited time is currently focused elsewhere. The oslo.db change is available at [1]. We'd like this to be merged in the next month but we can push that out to later in the cycle if teams need more time. Just shout. Please let us know if you have any concerns or if the above changes are not sufficient as a guide for how to address these issues. Cheers, Stephen [1] https://review.opendev.org/c/openstack/oslo.db/+/798136

4 4

[neutron] CI meeting agenda for 19.04
by Slawek Kaplonski 19 Apr '22

19 Apr '22

Hi, Just quick reminder that today at 15:00 UTC we will have Neutron CI weekly meeting. It will be the video meeting this week: *https://meetpad.opendev.org/neutron-ci-meetings[1]* Agenda for the meeting is available on https://etherpad.opendev.org/p/neutron-ci-meetings#L10[2] -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://meetpad.opendev.org/neutron-ci-meetings [2] https://etherpad.opendev.org/p/neutron-ci-meetings#L10

1 0

[neutron][nova] port binding fails for existing networks
by Eugen Block 19 Apr '22

19 Apr '22

Hi *, I have a kind of strange case which I'm trying to solve for hours, I could use some fresh ideas. It's a HA cloud (Victoria) deployed by Salt and the 2 control nodes are managed by pacemaker, the third controller will join soon. There are around 16 compute nodes at the moment. This two-node-control plane works well, except if there are unplanned outages. Since the last outage of one control node we struggle to revive neutron (I believe neutron is the issue here). I'll try to focus on the main issue here, let me know if more details are required. After the failed node was back online all openstack agents show as "up" (openstack compute service list, openstack network agent list). Running VMs don't seem to be impacted (as far as I can tell). But we can't create new instances in existing networks, and since we use Octavia we also can't (re)build any LBs at the moment. When I create a new test network the instance spawns successfully and is active within a few seconds. For existing networks we get the famous "port binding failed" from nova-compute.log. But I see the port being created, it just can't be attached to the instance. One more strange thing: I don't see any entries in the nova-scheduler.log or nova-conductor.log for the successfully built instance, except for the recently mentioned etcd3gw message from nova-conductor, but this didn't impact the instance creation yet. We have investigated this for hours, we have rebooted both control nodes multiple times in order to kill any remaining processes. The galera DB seems fine, rabbitmq also behaves normally (I think), we tried multiple times to put one node in standby to only have one node to look at which also didn't help. So basically we restarted everything multiple times on the control nodes and also nova-compute and openvswitch-agent on all compute nodes, the issue is still not resolved. Does anyone have further ideas to resolve this? I'd be happy to provide more details, just let me know what you need. Happy Easter! Eugen

3 5

OPS Meetup Meeting tomorrow
by Amy Marrich 19 Apr '22

19 Apr '22

Just a reminder that the OPS Meetup team will meet tomorrow morning at 13:00 UTC in the #openstack-operators room on OFTC. See you there!

1 0