- openstack-discuss - lists.openstack.org

[manila] CephFS NFS high availability cluster
by CHANU ROMAIN 13 Jun '22

13 Jun '22

Hello, I added Manila service with CephFS NFS driver to my openstack cluster. Everything works fine but I would like to add 2 nfs-ganesha servers to ensure high availability to the service. I configured haproxy to forward 2049 to ganesha backend but Manila cephFS NFS provides only IP restriction and see only haproxy's IP address. To make it work you have to add haproxy to allowed ip but it means everyone can access the share. So currently the only way I found out is to use pacemaker to set public vip to a running nfs-ganesha node. Could you confirm that is not possible to provide an active/active nfs-ganesha cluster with manila cephfs NFS driver? Best regards, Romain

2 1

[Magnum resize Master node root volume/docker volume or resize flavor]
by Nabeel Tariq Khan 13 Jun '22

13 Jun '22

Hi, Is it possible to resize the root volume or docker volume or resize the flavor of master or minion node on openstack magnum? regards, Nabeel Tariq Khan

1 0

Regarding Policy.json entries for glance image update not working for a user
by Adivya Singh 13 Jun '22

13 Jun '22

Hi Team, I have a use case where I have to give a user restriction on updating the image properties as a member. I have created a policy Json file and give the modify_image rule to the particular role, but still it is not working "modify_image": "role:user", This role is created in OpenStack. but still it is failing while updating properties with a particular user assigned to a role as "access denied" and unauthorized access Regards Adivya Singh

3 3

[ironic] No weekly meeting today (June 13)
by Iury Gregory 13 Jun '22

13 Jun '22

Hello Ironicers, I'm still on my way back home from the OIS, I'll not be able to run the upstream meeting I've asked in the irc and people are ok with skipping today's meeting, see you next week! -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic PTL * *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[neutron][networking-l2gw][puppet] Status of networking-l2gw project
by Takashi Kajinami 13 Jun '22

13 Jun '22

Hello team, # I understand networking-l2gw is not under neutron stadium # but adding [neutron] tag hoping it would attract more attention. We found a recent change in neutron[1] broke networking-l2gw[2], and some jobs in puppet repos and rdo, with l2gw enabled, are failing. I proposed a potential fix for the bug[3], but I noticed the project looks unmaintained according to the following points. - No change has been merged for more than 1 year - No release was created since Wallaby - The zuul setting still points the victoria job template (openstack-python3-victoria-jobs-neutron) I'm wondering if anybody is actually interested in maintaining the project. If not, I'll propose retiring the plugin support from puppet (and TripleO because it requires puppet implementation). I appreciate any feedback about this. [1] https://review.opendev.org/c/openstack/neutron/+/837392 [2] https://bugs.launchpad.net/networking-l2gw/+bug/1977980 [3] https://review.opendev.org/c/x/networking-l2gw/+/845141 Thank you, Takashi

2 1

Re: snapshot problem
by Eugen Block 12 Jun '22

12 Jun '22

Have you tried with debug logs? Has it worked with live snapshots before for other instances or did it never work and all snapshots were "cold"? Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: > Hi > kolla-ansible victoria version with ceph backend without volume > > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block <eblock(a)nde.ag> wrote: > >> Hi, >> >> can you share more details about your environment? Which openstack >> version is it? What is the storage backend? In earlier releases there >> was an option: >> >> #disable_libvirt_livesnapshot = false >> >> but this option has been deprecated. But if you're on an older version >> that could explain it. >> >> Zitat von Parsa Aminian <p.aminian.server(a)gmail.com>: >> >> > When I snapshot from the instance , server will gone away and its not >> > reachable until the snapshot is complete here is the logs : >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> instance >> > has a pending task (image_pending_upload). Skip. >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> 93fb420b3c604d4fae408b81135b58e9 >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image >> > upload >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> >> >>

1 0

[nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox
by Dave Hall 12 Jun '22

12 Jun '22

Hello, I'm trying to run a test/learning cluster on VirtualBox VMs deployed with Openstack-Ansible prior to deploying on real hardware. The playbooks succeed, and the basic functional checks all look good. However, I am unable to deploy an instance due to 'No Valid Host Found'. I have seen that this might be because VirtualBox does not support KVM. But I am also perplexed - I am able to deploy instances on an AIO running in VirtualBox on the same physical host. If there is an easy fix for this please point me in the right direction. If I need to destroy and re-clone my VMs I can try that as well. In an effort to learn, though, I'd really like to learn how to adjust the current deployment. Thanks. -Dave -- Dave Hall Binghamton University kdhall(a)binghamton.edu

2 5

snapshot problem
by Parsa Aminian 12 Jun '22

12 Jun '22

When I snapshot from the instance , server will gone away and its not reachable until the snapshot is complete here is the logs : 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting 2022-06-12 09:42:34.755 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) 2022-06-12 09:42:34.995 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance has a pending task (image_pending_upload). Skip. 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver [req-e79e4177-4712-4795-91da-853bc524fac0 93fb420b3c604d4fae408b81135b58e9 ef940663426b4c87a751afaf13b758e0 - default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image upload 2022-06-12 09:43:08.778 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event)

2 1

[all][tc] What's happening in Technical Committee: summary 10 June 2022: Reading: 5 min
by Ghanshyam Mann 10 Jun '22

10 Jun '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We cancelled this week's meeting. The next TC weekly meeting will be on 16 June Thursday at 15:00 UTC, feel free to add the topic to the agenda[1] by 15 June. 2. What we completed this week: ========================= * Added community infrastructure contributor help in upstream investment[2] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[3], many items are in-progress. Open Reviews ----------------- * Five open reviews for ongoing activities[4]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian proposal for ""SLURP release notes approach is up for review[5]. Improve project governance --------------------------------- Slawek has the framework proposal up and is under review[6]. New ELK service dashboard: e-r service ----------------------------------------------- No update this week. Consistent and Secure Default RBAC ------------------------------------------- We called for the operator's feedback on RBAC new directions, I sent an email about it[7] and also created etherpad to collect feedback in a central place[8]. My humble request is to use this central etherpad for feedback/discussion otherwise it might be difficult for me to collect/know feedback conclusions from a lot of 100s places. We have feedback from the berlin summit but we still encourage operators/local events or so keep adding the feedback and your view on RBAC. Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[9]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[10]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[11][12]. Project updates ------------------- * None 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[13]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [14] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/843749 [3] https://etherpad.opendev.org/p/tc-zed-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [6] https://review.opendev.org/c/openstack/governance/+/839880 [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028878.html [8] https://etherpad.opendev.org/p/rbac-operator-feedback [9] https://review.opendev.org/c/openstack/governance/+/836888 [10] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [11] https://etherpad.opendev.org/p/zuul-config-error-openstack [12] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [13] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [14] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Openstack-Ansible vs Kolla-Ansible -Xena or later
by Dave Hall 10 Jun '22

10 Jun '22

Hello, My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am sensitive to the effort that has been put into both of these projects, what I really need right now is the most fool-proof way to deploy and manage a small production cluster for academic instructional use. I have been working for a couple months with OpenStack-Ansible to deploy a small (3-node) test cluster on VirtualBox VMs in preparation for a larger deployment on real hardware - 6 to 10 nodes. (Details: Debian 11 deployment, Debian 10 infra, compute, and storage nodes) It has been slow going, at least partially due to some issues and limitations with VirtualBox. However, deploying a test cluster on VMs still seems preferable to just diving deployment on real hardware and going through multiple scrubs/reinstalls. Today I saw that Mark asked a question about Kolla-Ansible. So, as a 'beginner', should I shift and look at Kolla-Ansible, or should I stay the course and continue with Openstack-Ansible? What are the pros and cons of each? Thanks. -Dave -- Dave Hall Binghamton University kdhall(a)binghamton.edu

6 10

[TripleO] move stable/victoria tripleo to End Of Life OK?
by Marios Andreou 10 Jun '22

10 Jun '22

Hello TripleO the tripleo-ci team proposes that we transition the stable/victoria branch across all tripleo repos [1] to End Of Life [2]. Hopefully this is not a suprise to anyone as it has been discussed for a while (most recent discussion I can point to is at Zed PTG [3]). The stable/victoria branch was moved to Extended Maintenance with [4] so we can no longer make any releases for these repos. Victoria is not an active branch for TripleO (not an FFU branch) so we are just merging patches there on the way back to train. Are there any concerns, objections or any other comment? I'll give this a few days and if there are no further comments I'll post the EOL proposal and update here towards the end of next week, regards, marios [1] https://releases.openstack.org/teams/tripleo.html#victoria [2] https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… [3] https://etherpad.opendev.org/p/tripleo-zed-ci-load [4] https://review.opendev.org/c/openstack/releases/+/837982

1 2

[doc] question/enhancement-request on Sphinx theme used by openstack docs
by Waines, Greg 10 Jun '22

10 Jun '22

Hey ... I am on the technical steering committee for the openstack starlingx team and also provide primary technical review for our docs at https://docs.starlingx.io/ . docs.starlingx.io basically uses the openstack docs team's Sphinx theme for building our html docs ... see https://opendev.org/starlingx/docs . docs.starlingx.io is a very large doc suite. We are interested in improvements to the Table-of-Contents (TOC) navigation and presentation ... but we have limited Sphinx expertise. Specifically we are looking for the TOC support to have - more subsection header levels in TOC // such that user has full view of where he currently is in the doc hierarchy - highlighting to indicate currently selected section - separate scroll bar for TOC // ... so you don't lose sight of selected TOC selection when you scroll down on the main page which has large content. Have you ever considered such doc improvements? Did you have Sphinx expertise to do such improvements ? let us know, Greg.

2 1

[nova] local LVM volume on compute hosts
by Paul Harrison 10 Jun '22

10 Jun '22

I have installed a small test cluster using openstack-ansible (xena) and would like to use local LVM volumes for storing instance volumes - and following https://docs.openstack.org/nova/latest/configuration/config.html and (e.g. https://cloudnull.io/2017/12/nova-lvm-an-iop-love-story/) I have the libvirt section of my /etc/nova/nova.conf as below. -8<--- [libvirt] inject_partition = -2 inject_password = False inject_key = False virt_type = kvm live_migration_with_native_tls = true live_migration_scheme = tls live_migration_inbound_addr = 172.29.236.5 hw_disk_discard = ignore disk_cachemodes = none images_type = lvm images_volume_group = nova volume_clear = zero -8<—— however, the nova service continues to use cinder volumes from my storage host to store instances. Is there something else that needs to be configured to make this work, or is it obsolete?

2 7

[magnum] Docker Swarm errors on Yoga Ubuntu using Install Guide cut-n-paste
by Vince Mulhollon 10 Jun '22

10 Jun '22

I'm trying to use Magnum on a fresh Yoga install on Ubuntu to run some non-load balanced docker swarm workloads. Its a long story but the parts of the swarm workload are heterogeneous so a cluster would just get in the way. Everything else about the cluster is working fine, all is well, etc. I'm a small time VMware and OS admin with years of experience but absolute zero experience with Magnum. Also I have "some" experience with Docker Swarm clusters mostly created and assembled with Ansible. So I know quite well how to look at log files or configs in a general sense, but I have no idea what to expect from Magnum specifically as a "glue" technology. I've followed every detail of the "Magnum Installation Guide" for Yoga as seen at: https://docs.openstack.org/magnum/yoga/install/index.html When provisioning a docker swarm, I very literally cut and paste from: https://docs.openstack.org/magnum/yoga/install/launch-instance.html#provisi… cluster creation fails every time with "Property allowed_cidrs not assigned" exactly as seen on this mailing list post from 2021-August-20 about ten months ago on a Wallaby install, they were also using Ubuntu. http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024273.h… Advice at that point ranged from, try installing Octavia, which I was trying to avoid and I definitely do NOT want a load balancer in front of that particular workload, to Docker swarm is no longer tested for Magnum and it's in practice a K8S exclusive product. I played around for an afternoon and got nowhere WRT configuring the allowed cidrs for a k8s load balancer on a config that does not use a load balancer and does not use k8s. Heat works fine, its super fun, and one alternative is I could give up on Magnum and use some Heat templates, which I know how to do and seems simpler than getting Magnum to work at this point. Or I could brave the scary looking mostly-undocumented world of kuryr and zun, which I was attempting to avoid by using Magnum which is at least simple and easy to install. Or just keep spinning up manual images and letting ansible turn them into docker swarms for me. But, ideally, I'd like to use Magnum to spin up some Docker Swarms for personal curiosity reasons, if nothing else, it would be very convenient if it worked. So the point of this long ramble is, can anyone successfully running Docker Swarm on Yoga on Ubuntu using a very typical "Install Guide" installation provide a pointer or link or github or something that Google searches is not finding to get it to work? Also I hate to piggyback multiple questions but this paragraph also interests me, does anyone have any information and experience with this: https://docs.openstack.org/magnum/yoga/user/index.html#heat-stack-templates "Sample cluster driver To help developers in creating new COE drivers, a minimal cluster driver is provided as an example. The ‘docker’ cluster driver will simply deploy a single VM running Ubuntu with the latest Docker version installed. It is not a true cluster, but the simplicity will help to illustrate the key concepts. To be filled in" That "To be filled in" part would also be very interesting to me as worst case I could shove the Docker Swarm's workload onto a single Docker host, although then I'm back to orchestrating with Heat and presumably Ansible but via a different path.

1 0

[kolla-ansible][xena] Cell database HA/redundancy?
by Norrie, Andrew 10 Jun '22

10 Jun '22

Hi, We are currently planning some large OpenStack deployments utilizing kolla-ansible and I'm curious what folks are doing with the cell database HA/redundancy. With kolla-ansible (xena) it appears that a loadbalancer setup is only allowed for the default database shard (shard 0)... reference: https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb… If we are setting up separate cell database shards with Galera, I'm wondering if there is a convenient work-around or configuration for implementation of HA for these cell databases. In the inventory group_vars directory you can specify the group variables (for each cell database) as: nova_cell_database_address: nova_cell_database_group: but these aren't virtual IP hosts/addresses (non default db shard). This works perfectly fine with a single server cell database but not Galera. If that db server goes down the cell instance information is lost. Many thanks ... Andrew ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately."

3 3

Error creating octavia amphora
by Russell Stather 10 Jun '22

10 Jun '22

Hi I am seeing the below error when creating a load balancer. The security group does exist, and it is the correct group (tagged with charm-octavia) What am I missing here to resolve this? 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', 'message': 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server

4 6

[ops] details for Ops Meetup, Berlin, Tomorrow, Friday 2022-6-10
by Chris Morgan 09 Jun '22

09 Jun '22

A brief twitter thread here https://twitter.com/osopsmeetup/status/1535004027032940544?s=20&t=AgLzV6Qvg… has some reminders about tomorrow's event. Looking forward to seeing everyone. Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[CLOUDKITTY][OpenInfra summit 2022][June 9th] Presentation on Ceilometer, Gnocchi, and CloudKitty
by Rafael Weingärtner 09 Jun '22

09 Jun '22

Hello CloudKitters! It was a great opportunity to be able to present this talk for you all guys. It was a pleasure to see that so many people are interested in CloudKitty, Ceilometer, and Gnocchi. I really hope that we can all collaborate together to keep improving CloudKitty, and other related systems. Attached is the presentation we used during the presentation, so you all can consult this material. Also, as always, if you need some guidance, let us know. Furthermore, as I mentioned in the presentation, I will provide a step-by-step process on how to create a new metric using Ceilometer Dynamic pollster that can collect/generate/derive (however you want to call it) IOPs information for volumes. The new metric name is going to be called "dynamic_pollster.volume.services.volume.iops". Also, the idea is that we collect some attributes, and we use the attribute called "bootable", which indicates if the volume is bootable or not. Then, if the volume is bootable, we will charge for its IOPS; otherwise, we do not charge it. As I mentioned, this is just a very simple use case. It is not targeting anyone's need in production, but it can be used to get you going quite easily with the pipeline. Requirements: - Ceilometer up and running. Version 15.0.0 (or higher is recommended); - Gnocchi up and running. Version 4.3.0 (or higher). If you want to run this in production, this patch might be very important ( https://github.com/gnocchixyz/gnocchi/pull/1059) it has not been merged yet; - CloudKitty up and running. Version 10.0.0 (or higher). With all systems up and running, you will need to do the following: - Ceilometer - Add the new metric collection in the "polling.yaml" file. You can add it in any of the sections you already have there. Or, you can create its own, as follows: ``` - name: volume_iops_metric interval: 600 meters: - dynamic_pollster.volume.services.volume.iops ``` - After declaring the metrics there, you can create the pollster in the "pollster.d" folder in Ceilometer. The name of the file can be anything, as long as it ends with ".yaml"; for instance, "pollsters.d/volume-iops.yaml". Then, you will need to add the following content. Remember to replace the contents of "<>" with the values of your environment. ``` - name: "dynamic_pollster.volume.services.volume.iops" response_entries_key: "volumes" next_sample_url_attribute: "volumes_links | filter(lambda v: v.get('rel') == 'next', value) | list(value) | value[0] | value.get('href')" sample_type: "gauge" unit: "GB" value_attribute: "volume_type" endpoint_type: "volume" url_path: "/v3/<SERVICE_UUID_FOR_CEILOMETER_USER>/volumes/detail?all_tenants=true&limit=10" headers: "Openstack-API-Version": "latest" project_id_attribute: "os-vol-tenant-attr:tenant_id" metadata_fields: - "status" - "name" - "volume_type" - "availability_zone" - "user_id" - "bootable | str(value)" - "attachments | value or [ { 'server_id': '' } ] | value[0] | value['server_id']" metadata_mapping: "availability_zone": "dynamic_availability_zone" "name": "dynamic_display_name" "volume_type": "dynamic_volume_type" "attachments | value or [{ 'server_id': '' }] | value[0] | value['server_id']": "dynamic_instance_id" "bootable | str(value)": "bootable" value_mapping: "<VOLUME_TYPE_UUID_OR_VOLUME_TYPE_NAME>": "<THE IOPS THAT THIS VOLUME TYPE REPRESENTS>" ``` - After creating that, you will need to customize the file "gnocchi_resources.yaml", which is the file that maps metrics to resources, and attributes that are pushed to the backend. You can also create your own "gnocchi_resources.yaml" file, and then use it in the pipeline configuration in the "pipeline.yaml" file, in the configuration of "publishers". Then, you would have something like "gnocchi://?resources_definition_file=%2Fetc%2Fceilometer%2Fcustom_gnocchi_resources.yaml", assuming the file is called "custom_gnocchi_resources.yaml". In this file, you will need to add the new metric and attribute to be pushed to the metrics backend. The following is an example of "volume" resource declaration in the "gnocchi_resource.yaml" file with the new metric and attribute that we want to capture. ``` - resource_type: volume metrics: <MANY OTHER METRICS HERE> dynamic_pollster.volume.services.volume.iops: attributes: display_name: resource_metadata.(dynamic_display_name|notification_display_name) bootable: resource_metadata.bootable <MANY OTHER ENTRIES HERE> ``` - After the above configuration, you will need to create the new attribute in the Gnocchi resource type. To achieve that, you can issue the following command. ``` gnocchi resource-type update -a bootable:string:false volume ``` - Then, you will need to restart the Ceilometer. - Now, you can issue "gnocchi resource show <VOLUME ID>", and you should see the new metric in the resource, and the new attribute as well. With Ceilometer and Gnocchi configured for the new metric, we can move on to CloudKitty. - The new metric needs to be declared in the "metrics.yml" file. The content should be something similar to the following (adapt as you need). ``` dynamic_pollster.volume.services.volume.iops: unit: IOPS alt_name: disk-volume-iops-usage-hours groupby: - id - user_id - project_id - volume_type - display_name - revision_start - instance_id - availability_zone - bootable metadata: [] extra_args: aggregation_method: max resource_type: volume use_all_resource_revisions: false ``` - Restart CloudKitty processor - With the new metric introduced in CloudKitty processor, you will need to create the rating configuration. In our case, we wanted to rate the volumes marked as bootable for IOPs. The volumes that are not bootable would not be rated. The following are the steps one needs to execute to create the hashmap rating rules. - Create the service ``` openstack rating hashmap service create disk-volume-iops-usage-hours ``` - Then, create the field ``` openstack rating hashmap field create <SERVICE_ID> <FIELD NAME WE WANT TO RATE, IN OUR CASE, IT IS bootable> ``` - After that, we can create the rating rule ``` openstack rating hashmap mapping create --field <FIELD UUID THAT WE CREATE> --value 'true' 1 --type flat ``` - Having done that, you should be able to rate all volumes by IOPS if they have the flag "bootable" set to true. Otherwise, they will not be rated. That is all that one needs to execute to replicate the example I showed during the presentation. If you guys have any doubts or difficulties when using CloudKitty, do not hesitate to ping me. -- Rafael Weingärtner

1 0

[rbac][nova][cinder] Canned roles for service users / inter-service communication (e.g. event submission)
by Christian Rohmann 09 Jun '22

09 Jun '22

Hey openstack-discuss, I posted to the ML quite a while ago about an issue of resized (Cinder) volumes not being propagated to the (Nova) instance. See http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020476…. The issue there was Cinder being not allowed to send the "volume-extended" event (or any event for that matter via the Nova API just using the user token. For this a configurable additional "privileged user" was added to the config quite a while back with https://opendev.org/openstack/cinder/commit/04003d7c513ed4dd5129cbd5ad1af14…. While the functionality then works I suppose there should be canned and maintained RBAC roles for such kind of inter service to service communications, e.g. to emit events to others. Otherwise deployments likely will use admin privileged users ignoring the least privilege principle and creating an large attack surface. And there are quite few of those relations even with the most commonly used services. Cinder -> nova, nova-> cincer, nova->ironic, .... nova-> neutron, .... Are such canned RBAC rules for "special" inter service users on the backlog somewhere? Or am I totally misconceiving the issue here? I know there is https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… and also the question for feedback at https://etherpad.opendev.org/p/rbac-operator-feedback, but that all seems to focus on the impact of roles used by humans / users and not about service roles at all. Regards Christian

2 3

[all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC)
by Ghanshyam Mann 09 Jun '22

09 Jun '22

Hello Everyone, As you might know, we are redesigning the OpenStack default RBAC. The new design target two things: 1. 'new defaults (reader role)' 2. "Scope" concept It is hard to explain the details in email but the below doc is a good place to start understanding this: - https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… We as a community think 1st target (reader role) is a good thing to do and it will definitely be useful in many cases. But we need feedback on the "Scope" concept. To understand what it is and how it can impact your existing use case/deployment, please ref the documentation mentioned in the etherpad[1] (if there is any question about its design/usage we are planning, feel free to reply here or contact us in #openstack-tc IRC channel). * If you are an operator, we really need your feedback if the 'Scope' concept is a useful thing for your deployment/use-case or not. * If you are attending events have operators also attending (for example, project operator feedback (like nova[2]), forum sessions in berlin summit, ops meetup or any local operator event), please communicate about the required feedback. * Due to various reasons, many of us involved in RBAC work are not travelling to Berlin and we have this topic to be discussed in Berlin ops meetup[3] but we require someone knowing RBAC new design moderate this topic. Please reach out to us if you would like to help. Central Etherpad to collect feedback (this can be used to collect from various forums/places): * https://etherpad.opendev.org/p/rbac-operator-feedback [1] https://etherpad.opendev.org/p/rbac-operator-feedback [2] https://etherpad.opendev.org/p/nova-berlin-meet-and-greet [3]https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L74 -gmann

4 12

[tripleo] gate blocker - master/wallaby - "SELinux boolean os_enable_vtpm does not exist."
by Ronelle Landy 09 Jun '22

09 Jun '22

Hello All, We have a consistent failure on multiple jobs running on check/gate/integration lines on master and wallaby changes - which shows up as a deployment error: FATAL | Enable os_enable_vtpm SELinux boolean for vTPM | standalone | error={"changed": false, "msg": "SELinux boolean os_enable_vtpm does not exist."} Details of the failure are in: https://bugs.launchpad.net/tripleo/+bug/1977873 We are investigating a fix/workaround. Please hold rechecks - we will post out again with updates. Thank you!

1 3

[kolla] restart mariadb container on host 1
by Gregory Orange 09 Jun '22

09 Jun '22

Hi all, I wanted to temporarily change a mariadb config in /etc/kolla/mariadb/galera.cnf so I intended to stop mariadb on host 1, make the change, start it again, then repeat in turn for hosts 2 and 3. Upon starting it on host 1, it doesn't rejoin, but creates its own separate cluster. Dutiful really, since logs show: WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster How do I avoid that? General advice on handling this kind of activity would also be welcome. Thank you, Greg.

2 2

[kolla-ansible][kolla] merge_configs module replacing colon to equal
by Sharath Ck 09 Jun '22

09 Jun '22

Hi, In kolla-ansible, I am using the merge_configs module to merge service configs files. I want to merge api-paste.ini config with my custom changes, however while merging if a line consists of one colon, it is getting replaced with equal sign. merge_configs splits the line by considering colon as delimiter and treats two parts as key and value. Later which will be substituted as key = value. Original */v2.1: oscomputeversion_v2* Merged */v2.1= oscomputeversion_v2 * Below, colon will be 1 and equal will be -1, *def _split_key_value(self, line): colon = line.find(':') equal = line.find('=') if colon < 0 and equal < 0: return self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and equal < colon): key, value = line[:equal], line[equal + 1:] else: key, value = line[:colon], line[colon + 1:] value = value.strip() if value and value[0] == value[-1] and value.startswith(("\"", "'")): value = value[1:-1] return key.strip(), [value]* I want to keep the config as it is and merge only my custom changes. Please let me know how to achieve this or colon can be escaped somehow. Regards, Sharath Regards, Sharath

3 3

[CLOUDKITTY] presentation used in the forum panels
by Rafael Weingärtner 08 Jun '22

08 Jun '22

Hello CloudKitters! It was a pleasure to meet you guys during the OpenStack summit. Also, it was a good surprise to see that so many people already use CloudKitty. I really hope that we can all collaborate together to keep improving CloudKitty, and making it more and more solid. Attached is the presentation we used during the Forum, so you all can consult this material. Also, as always, if you need some guidance/help/or anything else, let us know. Link for the presentation online: https://docs.google.com/presentation/d/15e37xGKCJoTMwbarrGcSlLpECUkcJ2exazW… -- Rafael Weingärtner

2 1

Re: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Lokendra Rathour 08 Jun '22

08 Jun '22

Hi Swogat, Thanks once again. with the files as shown below I am running the overcloud deploy for wallaby using this command: (undercloud) [stack@undercloud ~]$ cat deploy_overcloud_working_1.sh openstack overcloud deploy --templates \ -n /home/stack/templates/network_data.yaml \ -r /home/stack/templates/roles_data.yaml \ -e /home/stack/templates/environment.yaml \ -e /home/stack/templates/environments/network-isolation.yaml \ -e /home/stack/templates/environments/network-environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml \ -e /home/stack/templates/ironic-config.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ -e /home/stack/containers-prepare-parameter.yaml (undercloud) [stack@undercloud ~]$ This deployment is on ipv6 using triple0 wallaby, templates, as mentioned below, are generated using rendering steps and the network_data.yaml the roles_data.yaml Steps used to render the templates: cd /usr/share/openstack-tripleo-heat-templates/ ./tools/process-templates.py -o ~/openstack-tripleo-heat-templates-rendered_at_wallaby -n /home/stack/templates/network_data.yaml -r /home/stack/templates/roles_data.yaml *Now if i try adding the related to VIP port I do get the error as:* 2022-05-30 10:37:12.792 979387 WARNING tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template to file: /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/controller-role.yaml 2022-05-30 10:37:12.792 979387 WARNING tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template to file: /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/compute-role.yaml 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: ValueError: The environment is not a valid YAML mapping data type. 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last): 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1189, in take_action 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud stack, parsed_args, new_tht_root, user_tht_root) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 227, in create_env_files 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud created_env_files, parsed_args, new_tht_root, user_tht_root) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 204, in build_image_params 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud cleanup=(not parsed_args.no_cleanup)) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1929, in process_multiple_environments 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud env_path=env_path, include_env_in_files=include_env_in_files) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", line 326, in process_environment_and_files 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud env = environment_format.parse(raw_env) 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/heatclient/common/environment_format.py", line 50, in parse 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise ValueError(_('The environment is not a valid ' 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud ValueError: The environment is not a valid YAML mapping data type. 2022-05-30 10:37:14.455 979387 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud 2022-05-30 10:37:14.457 979387 ERROR openstack [-] The environment is not a valid YAML mapping data type. 2022-05-30 10:37:14.457 979387 INFO osc_lib.shell [-] END return value: 1 (undercloud) [stack@undercloud ~]$ This is more of a syntax error where it is not able to understand the passed VIP data file: undercloud) [stack@undercloud ~]$ cat /home/stack/templates/vip-data-default-network-isolation.yaml - dns_name: overcloud network: internal_api - dns_name: overcloud network: external - dns_name: overcloud network: ctlplane - dns_name: overcloud network: oc_provisioning - dns_name: overcloud network: j3mgmt Please advise, also please note that similar templates generated in prior releases such as train/ussuri works perfectly. Please check the list of *templates *files: drwxr-xr-x. 2 stack stack 68 May 30 09:22 environments -rw-r--r--. 1 stack stack 265 May 27 13:47 environment.yaml -rw-rw-r--. 1 stack stack 297 May 27 13:47 init-repo.yaml -rw-r--r--. 1 stack stack 570 May 27 13:47 ironic-config.yaml drwxrwxr-x. 4 stack stack 4096 May 27 13:53 network -rw-r--r--. 1 stack stack 6370 May 27 14:26 network_data.yaml -rw-r--r--. 1 stack stack 11137 May 27 13:53 roles_data.yaml -rw-r--r--. 1 stack stack 234 May 30 09:23 vip-data-default-network-isolation.yaml (undercloud) [stack@undercloud templates]$ cat environment.yaml parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 3 ComputeCount: 1 TimeZone: 'Asia/Kolkata' NtpServer: ['30.30.30.3'] NeutronBridgeMappings: datacentre:br-tenant NeutronFlatNetworks: datacentre (undercloud) [stack@undercloud templates]$ (undercloud) [stack@undercloud templates]$ cat ironic-config.yaml parameter_defaults: NovaSchedulerDefaultFilters: - AggregateInstanceExtraSpecsFilter - AvailabilityZoneFilter - ComputeFilter - ComputeCapabilitiesFilter - ImagePropertiesFilter IronicEnabledHardwareTypes: - ipmi - redfish IronicEnabledPowerInterfaces: - ipmitool - redfish IronicEnabledManagementInterfaces: - ipmitool - redfish IronicCleaningDiskErase: metadata IronicIPXEEnabled: true IronicInspectorSubnets: - ip_range: 172.23.3.100,172.23.3.150 (undercloud) [stack@undercloud templates]$ cat network_data.yaml - name: J3Mgmt name_lower: j3mgmt vip: true vlan: 400 ipv6: true ipv6_subnet: 'fd80:fd00:fd00:4000::/64' ipv6_allocation_pools: [{'start': 'fd80:fd00:fd00:4000::10', 'end': 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] mtu: 9000 - name: InternalApi name_lower: internal_api vip: true vlan: 418 ipv6: true ipv6_subnet: 'fd00:fd00:fd00:2000::/64' ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:2000::10', 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] mtu: 9000 - name: External vip: true name_lower: external vlan: 408 ipv6: true gateway_ipv6: 'fd00:fd00:fd00:9900::1' ipv6_subnet: 'fd00:fd00:fd00:9900::/64' ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:9900::10', 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] mtu: 9000 - name: OCProvisioning vip: true name_lower: oc_provisioning vlan: 412 ip_subnet: '172.23.3.0/24' allocation_pools: [{'start': '172.23.3.10', 'end': '172.23.3.50'}] mtu: 9000 (undercloud) [stack@undercloud templates]$ cat roles_data.yaml ############################################################################### # File generated by TripleO ############################################################################### ############################################################################### # Role: Controller # ############################################################################### - name: Controller description: | Controller role that has all the controller services loaded and handles Database, Messaging, and Network functions. CountDefault: 1 tags: - primary - controller # Create external Neutron bridge for SNAT (and floating IPs when using # ML2/OVS without DVR) - external_bridge networks: External: subnet: external_subnet InternalApi: subnet: internal_api_subnet OCProvisioning: subnet: oc_provisioning_subnet J3Mgmt: subnet: j3mgmt_subnet # For systems with both IPv4 and IPv6, you may specify a gateway network for # each, such as ['ControlPlane', 'External'] default_route_networks: ['External'] HostnameFormatDefault: '%stackname%-controller-%index%' RoleParametersDefault: OVNCMSOptions: "enable-chassis-as-gw" # Deprecated & backward-compatible values (FIXME: Make parameters consistent) # Set uses_deprecated_params to True if any deprecated params are used. uses_deprecated_params: True deprecated_param_extraconfig: 'controllerExtraConfig' deprecated_param_flavor: 'OvercloudControlFlavor' deprecated_param_image: 'controllerImage' deprecated_nic_config_name: 'controller.yaml' update_serial: 1 ServicesDefault: - OS::TripleO::Services::Aide - OS::TripleO::Services::AodhApi - OS::TripleO::Services::AodhEvaluator .. . ..############################################################################### # Role: Compute # ############################################################################### - name: Compute description: | Basic Compute Node role CountDefault: 1 # Create external Neutron bridge (unset if using ML2/OVS without DVR) tags: - compute - external_bridge networks: InternalApi: subnet: internal_api_subnet J3Mgmt: subnet: j3mgmt_subnet HostnameFormatDefault: '%stackname%-novacompute-%index%' RoleParametersDefault: FsAioMaxNumber: 1048576 TunedProfileName: "virtual-host" # Deprecated & backward-compatible values (FIXME: Make parameters consistent) # Set uses_deprecated_params to True if any deprecated params are used. # These deprecated_params only need to be used for existing roles and not for # composable roles. uses_deprecated_params: True deprecated_param_image: 'NovaImage' deprecated_param_extraconfig: 'NovaComputeExtraConfig' deprecated_param_metadata: 'NovaComputeServerMetadata' deprecated_param_scheduler_hints: 'NovaComputeSchedulerHints' deprecated_param_ips: 'NovaComputeIPs' deprecated_server_resource_name: 'NovaCompute' deprecated_nic_config_name: 'compute.yaml' update_serial: 25 ServicesDefault: - OS::TripleO::Services::Aide - OS::TripleO::Services::AuditD - OS::TripleO::Services::BootParams (undercloud) [stack@undercloud templates]$ cat environments/network-environment.yaml #This file is an example of an environment file for defining the isolated #networks and related parameters. resource_registry: # Network Interface templates to use (these files must exist). You can # override these by including one of the net-*.yaml environment files, # such as net-bond-with-vlans.yaml, or modifying the list here. # Port assignments for the Controller OS::TripleO::Controller::Net::SoftwareConfig: OS::Heat::None # Port assignments for the Compute OS::TripleO::Compute::Net::SoftwareConfig: OS::Heat::None parameter_defaults: # This section is where deployment-specific configuration is done # ServiceNetMap: IronicApiNetwork: oc_provisioning IronicNetwork: oc_provisioning # This section is where deployment-specific configuration is done ControllerNetworkConfigTemplate: 'templates/bonds_vlans/bonds_vlans.j2' ComputeNetworkConfigTemplate: 'templates/bonds_vlans/bonds_vlans.j2' # Customize the IP subnet to match the local environment J3MgmtNetCidr: 'fd80:fd00:fd00:4000::/64' # Customize the IP range to use for static IPs and VIPs J3MgmtAllocationPools: [{'start': 'fd80:fd00:fd00:4000::10', 'end': 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] # Customize the VLAN ID to match the local environment J3MgmtNetworkVlanID: 400 # Customize the IP subnet to match the local environment InternalApiNetCidr: 'fd00:fd00:fd00:2000::/64' # Customize the IP range to use for static IPs and VIPs InternalApiAllocationPools: [{'start': 'fd00:fd00:fd00:2000::10', 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] # Customize the VLAN ID to match the local environment InternalApiNetworkVlanID: 418 # Customize the IP subnet to match the local environment ExternalNetCidr: 'fd00:fd00:fd00:9900::/64' # Customize the IP range to use for static IPs and VIPs # Leave room if the external network is also used for floating IPs ExternalAllocationPools: [{'start': 'fd00:fd00:fd00:9900::10', 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] # Gateway router for routable networks ExternalInterfaceDefaultRoute: 'fd00:fd00:fd00:9900::1' # Customize the VLAN ID to match the local environment ExternalNetworkVlanID: 408 # Customize the IP subnet to match the local environment OCProvisioningNetCidr: '172.23.3.0/24' # Customize the IP range to use for static IPs and VIPs OCProvisioningAllocationPools: [{'start': '172.23.3.10', 'end': '172.23.3.50'}] # Customize the VLAN ID to match the local environment OCProvisioningNetworkVlanID: 412 # List of Neutron network types for tenant networks (will be used in order) NeutronNetworkType: 'geneve,vlan' # Neutron VLAN ranges per network, for example 'datacentre:1:499,tenant:500:1000': NeutronNetworkVLANRanges: 'datacentre:1:1000' # Customize bonding options, e.g. "mode=4 lacp_rate=1 updelay=1000 miimon=100" # for Linux bonds w/LACP, or "bond_mode=active-backup" for OVS active/backup. BondInterfaceOvsOptions: "bond_mode=active-backup" (undercloud) [stack@undercloud templates]$ (undercloud) [stack@undercloud templates]$ cat environments/network-isolation.yaml # NOTE: This template is now deprecated, and is only included for compatibility # when upgrading a deployment where this template was originally used. For new # deployments, set "ipv6: true" on desired networks in network_data.yaml, and # include network-isolation.yaml. # # Enable the creation of Neutron networks for isolated Overcloud # traffic and configure each role to assign ports (related # to that role) on these networks. resource_registry: # networks as defined in network_data.yaml OS::TripleO::Network::J3Mgmt: ../network/j3mgmt_v6.yaml OS::TripleO::Network::InternalApi: ../network/internal_api_v6.yaml OS::TripleO::Network::External: ../network/external_v6.yaml OS::TripleO::Network::OCProvisioning: ../network/oc_provisioning.yaml # Port assignments for the VIPs OS::TripleO::Network::Ports::J3MgmtVipPort: ../network/ports/j3mgmt_v6.yaml OS::TripleO::Network::Ports::InternalApiVipPort: ../network/ports/internal_api_v6.yaml OS::TripleO::Network::Ports::ExternalVipPort: ../network/ports/external_v6.yaml OS::TripleO::Network::Ports::OCProvisioningVipPort: ../network/ports/oc_provisioning.yaml # Port assignments by role, edit role definition to assign networks to roles. # Port assignments for the Controller OS::TripleO::Controller::Ports::J3MgmtPort: ../network/ports/j3mgmt_v6.yaml OS::TripleO::Controller::Ports::InternalApiPort: ../network/ports/internal_api_v6.yaml OS::TripleO::Controller::Ports::ExternalPort: ../network/ports/external_v6.yaml OS::TripleO::Controller::Ports::OCProvisioningPort: ../network/ports/oc_provisioning.yaml # Port assignments for the Compute OS::TripleO::Compute::Ports::J3MgmtPort: ../network/ports/j3mgmt_v6.yaml OS::TripleO::Compute::Ports::InternalApiPort: ../network/ports/internal_api_v6.yaml parameter_defaults: # Enable IPv6 environment for Manila ManilaIPv6: True (undercloud) [stack@undercloud templates]$ On Tue, May 24, 2022 at 5:04 PM Lokendra Rathour <lokendrarathour(a)gmail.com> wrote: > Thanks, I'll check them out. > will let you know in case it works out. > > On Tue, May 24, 2022 at 2:37 PM Swogat Pradhan <swogatpradhan22(a)gmail.com> > wrote: > >> Hi, >> Please find the below templates: >> These are for openstack wallaby release: >> >> (undercloud) [stack@hkg2director workplace]$ cat custom_network_data.yaml >> - name: Storage >> name_lower: storage >> vip: true >> mtu: 1500 >> subnets: >> storage_subnet: >> ip_subnet: 172.25.202.0/26 >> allocation_pools: >> - start: 172.25.202.6 >> end: 172.25.202.20 >> vlan: 1105 >> - name: StorageMgmt >> name_lower: storage_mgmt >> vip: true >> mtu: 1500 >> subnets: >> storage_mgmt_subnet: >> ip_subnet: 172.25.202.64/26 >> allocation_pools: >> - start: 172.25.202.72 >> end: 172.25.202.87 >> vlan: 1106 >> - name: InternalApi >> name_lower: internal_api >> vip: true >> mtu: 1500 >> subnets: >> internal_api_subnet: >> ip_subnet: 172.25.201.192/26 >> allocation_pools: >> - start: 172.25.201.198 >> end: 172.25.201.212 >> vlan: 1104 >> - name: Tenant >> vip: false # Tenant network does not use VIPs >> mtu: 1500 >> name_lower: tenant >> subnets: >> tenant_subnet: >> ip_subnet: 172.25.202.128/26 >> allocation_pools: >> - start: 172.25.202.135 >> end: 172.25.202.150 >> vlan: 1108 >> - name: External >> name_lower: external >> vip: true >> mtu: 1500 >> subnets: >> external_subnet: >> ip_subnet: 172.25.201.128/26 >> allocation_pools: >> - start: 172.25.201.135 >> end: 172.25.201.150 >> gateway_ip: 172.25.201.129 >> vlan: 1103 >> >> (undercloud) [stack@hkg2director workplace]$ cat custom_vip_data.yaml >> - network: ctlplane >> #dns_name: overcloud >> ip_address: 172.25.201.91 >> subnet: ctlplane-subnet >> - network: external >> #dns_name: overcloud >> ip_address: 172.25.201.150 >> subnet: external_subnet >> - network: internal_api >> #dns_name: overcloud >> ip_address: 172.25.201.250 >> subnet: internal_api_subnet >> - network: storage >> #dns_name: overcloud >> ip_address: 172.25.202.50 >> subnet: storage_subnet >> - network: storage_mgmt >> #dns_name: overcloud >> ip_address: 172.25.202.90 >> subnet: storage_mgmt_subnet >> >> (undercloud) [stack@hkg2director workplace]$ cat >> overcloud-baremetal-deploy.yaml >> - name: Controller >> count: 4 >> defaults: >> networks: >> - network: ctlplane >> vif: true >> - network: external >> subnet: external_subnet >> - network: internal_api >> subnet: internal_api_subnet >> - network: storage >> subnet: storage_subnet >> - network: storage_mgmt >> subnet: storage_mgmt_subnet >> - network: tenant >> subnet: tenant_subnet >> network_config: >> template: /home/stack/templates/controller.j2 >> default_route_network: >> - external >> instances: >> - hostname: overcloud-controller-0 >> name: dc1-controller2 >> #provisioned: false >> - hostname: overcloud-controller-1 >> name: dc2-controller2 >> #provisioned: false >> - hostname: overcloud-controller-2 >> name: dc1-controller1 >> #provisioned: false >> - hostname: overcloud-controller-no-ceph-3 >> name: dc2-ceph2 >> #provisioned: false >> #- hostname: overcloud-controller-3 >> #name: dc2-compute3 >> #provisioned: false >> >> - name: Compute >> count: 5 >> defaults: >> networks: >> - network: ctlplane >> vif: true >> - network: internal_api >> subnet: internal_api_subnet >> - network: tenant >> subnet: tenant_subnet >> - network: storage >> subnet: storage_subnet >> network_config: >> template: /home/stack/templates/compute.j2 >> instances: >> - hostname: overcloud-novacompute-0 >> name: dc2-compute1 >> #provisioned: false >> - hostname: overcloud-novacompute-1 >> name: dc2-ceph1 >> #provisioned: false >> - hostname: overcloud-novacompute-2 >> name: dc1-compute1 >> #provisioned: false >> - hostname: overcloud-novacompute-3 >> name: dc1-compute2 >> #provisioned: false >> - hostname: overcloud-novacompute-4 >> name: dc2-compute3 >> #provisioned: false >> >> - name: CephStorage >> count: 4 >> defaults: >> networks: >> - network: ctlplane >> vif: true >> - network: internal_api >> subnet: internal_api_subnet >> - network: storage >> subnet: storage_subnet >> - network: storage_mgmt >> subnet: storage_mgmt_subnet >> network_config: >> template: /home/stack/templates/ceph-storage.j2 >> instances: >> - hostname: overcloud-cephstorage-0 >> name: dc2-controller1 >> #provisioned: false >> # - hostname: overcloud-cephstorage-1 >> # name: dc2-ceph2 >> - hostname: overcloud-cephstorage-1 >> name: dc1-ceph1 >> # provisioned: false >> - hostname: overcloud-cephstorage-2 >> name: dc1-ceph2 >> #provisioned: false >> - hostname: overcloud-cephstorage-3 >> name: dc2-compute2 >> #provisioned: false >> >> >> You must use these templates to provision network, vip and nodes. >> You must use the output files generated during the provisioning step in >> openstack overcloud deploy command using -e parameter. >> >> With regards, >> Swogat Pradhan >> >> >> On Mon, May 23, 2022 at 8:33 PM Lokendra Rathour < >> lokendrarathour(a)gmail.com> wrote: >> >>> Hi Swogat, >>> I tried checking your solution and my templates but could not relate >>> much. >>> But issue seems the same >>> >>> http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028401.html >>> >>> I tried somemore ways but looks like some issue with templates. >>> Can you please share the templates used to deploy the overcloud. >>> >>> Mysetup have 3 controller and 1 compute. >>> >>> Thanks once again for reading my mail. >>> >>> Waiting for your reply. >>> >>> -Lokendra >>> >>> On Fri, 20 May 2022, 08:25 Swogat Pradhan, <swogatpradhan22(a)gmail.com> >>> wrote: >>> >>>> Hi, >>>> Yes I was able to find the issue and fix it. >>>> The issue was with the overcloud-baremetal-deployed.yaml file i was >>>> trying to provision controller-0, controller-1 and controller-3 and kept >>>> controller-2 aside for later, but the tripleo scripts are built in such a >>>> way that they were taking controller- 0, 1 and 2 inplace of controller-3, >>>> so the network ports and vip were created for controller 0,1 and 2 but not >>>> for 3 , so this error was popping off. Also i would request you to check >>>> the jinja nic templates and once the node provisioning is done check the >>>> /etc/os-net-config/config.json/yaml file for syntax if using bonded nic >>>> template. >>>> If you need any more infor please let me know. >>>> >>>> With regards, >>>> Swogat Pradhan >>>> >>>> >>>> >>>> On Fri, May 20, 2022 at 8:01 AM Lokendra Rathour < >>>> lokendrarathour(a)gmail.com> wrote: >>>> >>>>> Hi Swogat, >>>>> Thanks for raising this issue. >>>>> Did you find any solution? to this problem ? >>>>> >>>>> Please let me know it might be helpful >>>>> >>>>> >>>>> On Tue, Apr 19, 2022 at 12:43 PM Swogat Pradhan < >>>>> swogatpradhan22(a)gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> I am currently trying to deploy openstack wallaby using tripleo arch. >>>>>> I created the network jinja templates, ran the following commands >>>>>> also: >>>>>> >>>>>> #openstack overcloud network provision --stack overcloud --output >>>>>> networks-deployed-environment.yaml custom_network_data.yaml >>>>>> # openstack overcloud network vip provision --stack overcloud >>>>>> --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>>> # openstack overcloud node provision --stack overcloud >>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>>> overcloud-baremetal-deploy.yaml >>>>>> >>>>>> and used the environment files in the openstack overcloud deploy >>>>>> command: >>>>>> >>>>>> (undercloud) [stack@hkg2director ~]$ cat deploy.sh >>>>>> #!/bin/bash >>>>>> THT=/usr/share/openstack-tripleo-heat-templates/ >>>>>> CNF=/home/stack/ >>>>>> openstack overcloud deploy --templates $THT \ >>>>>> -r $CNF/templates/roles_data.yaml \ >>>>>> -n $CNF/workplace/custom_network_data.yaml \ >>>>>> -e ~/containers-prepare-parameter.yaml \ >>>>>> -e $CNF/templates/node-info.yaml \ >>>>>> -e $CNF/templates/scheduler-hints.yaml \ >>>>>> -e $CNF/workplace/networks-deployed-environment.yaml \ >>>>>> -e $CNF/workplace/vip-deployed-environment.yaml \ >>>>>> -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ >>>>>> -e $CNF/workplace/custom-net-bond-with-vlans.yaml >>>>>> >>>>>> Now when i run the ./deploy.sh script i encounter an error stating: >>>>>> >>>>>> ERROR openstack [-] Resource >>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>> included with the deployment command.: >>>>>> tripleoclient.exceptions.InvalidConfiguration: Resource >>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>> included with the deployment command. >>>>>> 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return >>>>>> value: 1 >>>>>> >>>>>> Can someone tell me where the mistake is? >>>>>> >>>>>> With regards, >>>>>> Swogat Pradhan >>>>>> >>>>> >>>>> >>>>>

3 8

[cinder] Bug deputy report for week of 06-08-2022
by Sofia Enriquez 08 Jun '22

08 Jun '22

This is a bug report from 06-01-2022 to 06-08-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Low - https://bugs.launchpad.net/cinder/+bug/1976499 "[IBM Storwize] lsfcportsetmember is being called in the wrong SVC code level." Fix proposed to master. Wishlist - https://bugs.launchpad.net/cinder/+bug/1977868 "When create volume from image, image is fetched twice." Unassigned. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[Tripleo] - Provisioning Baremetal Instances in Wallaby Openstack
by Anirudh Gupta 08 Jun '22

08 Jun '22

Hi Harald/Julia/Openstack Team I had successfully installed Ironic Service in Tripleo Train, Ussuri Release with the help of openstack discuss team members. https://lists.openstack.org/pipermail/openstack-discuss/2022-February/02704… I had custom network configuration for which I had passed Service net map as below in network-environment file. parameter_defaults: ServiceNetMap: IronicApiNetwork: oc_provisioning IronicNetwork: oc_provisioning But with the Wallaby Release, since the deployment format has changed There is no network-environment file. So I tried passing it in a separate environment.yaml file along with the templates. Command to deploy in wallaby: openstack overcloud deploy --templates \ --networks-file /home/stack/templates/custom_network_data.yaml \ --vip-file /home/stack/templates/custom_vip_data.yaml \ --baremetal-deployment /home/stack/templates/overcloud-baremetal-deploy.yaml --deployed-server \ --network-config \ -e /home/stack/templates/environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml \ -e /home/stack/templates/ironic-config.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ -e /home/stack/containers-prepare-parameter.yaml Without Service Net Map, deployment was successful but TFTP of image was an issue since we are going with custom network approach. Passing Service Net map details in a seperate file environment.yaml, is giving the below error: 24d-f43f-6f76-000000006ce9 | TASK | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 2022-06-07 11:16:46.806982 | | WARNING | ERROR: Can't run container ironic_db_sync stderr: Error: statfs /var/lib/config-data/puppet-generated/ironic_api: no such file or directory 2022-06-07 11:16:46.809252 | 525400fd-f24d-f43f-6f76-000000006c79 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-1 | error={"changed": false, "msg": "Failed containers: ironic_db_sync"} 2022-06-07 11:16:46.810197 | 525400fd-f24d-f43f-6f76-000000006c79 | TIMING | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-1 | 0:11:40.574944 | 10.62s 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container ironic_db_sync stderr: Error: statfs /var/lib/config-data/puppet-generated/ironic_api: no such file or directory 2022-06-07 11:16:47.684455 | 525400fd-f24d-f43f-6f76-000000006ce9 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-2 | error={"changed": false, "msg": "Failed containers: ironic_db_sync"} Can you please suggest the place we can pass the service net map details in Wallaby Release to resolve this issue? Regards Anirudh Gupta

2 3

Using infiniband for openstack network communications
by A Monster 07 Jun '22

07 Jun '22

Is it possible to use the infiniband port for openstack networks without having to use the infiniband port as an ethernet port ?

4 3

[cinder][drivers] Register blueprints for your drivers
by Rajat Dhasmana 07 Jun '22

07 Jun '22

Hello Driver Maintainers, If you are planning to submit your driver in the Zed development cycle, make sure you have a blueprint registered against your driver. There is a doc available[1] describing how to register a blueprint and also information related to its attributes. Also take a look at the previously filed blueprints here[2] for reference. We had a discussion in the last mid cycle (1st June) and the following is a list of drivers that are currently proposed. If your driver is in the list or not, please file a blueprint and mention the driver patches in the work items section of the blueprint that will help us prioritize the review of that driver for the Zed development cycle. 1) DataCore (reintroduced), FC and iSCSI: https://review.opendev.org/c/openstack/cinder/+/836996 2) YADRO Tatlin.UNIFIED, FC and iSCSI: https://review.opendev.org/c/openstack/cinder/+/825492 3) Dell EMC PowerStore, NFS: https://review.opendev.org/c/openstack/cinder/+/797608 4) Pure Storage, NVMe-RoCE: https://review.opendev.org/c/openstack/cinder/+/799871 5) HPE XP Storage, FC and iSCSI: https://review.opendev.org/c/openstack/cinder/+/815582 [1] https://wiki.openstack.org/wiki/Blueprints [2] https://blueprints.launchpad.net/cinder Thanks and regards Rajat Dhasmana

1 0

[Octavia] Weekly meeting cancelled
by Gregory Thiemonge 07 Jun '22

07 Jun '22

Hi, This week's upstream meeting is cancelled, because it conflicts with the OpenInfra Summit (and I'm also on PTO), See you next week, Thanks, Gregory

1 0

[sdk] specify mac address when creating ports with nova
by jdratlif＠globalnoc.iu.edu 07 Jun '22

07 Jun '22

When using connection.compute.create_server(), what is the structure of the networks parameter? I see in the example is a list of objects with a single key in the list being the uuid of the network. I know there are other parameters that could be in this list though. The os-migrate ansible playbooks add fixed_ip to this. fixed_ip is not a Network or a Port class parameter. Port has fixed_ips, but that would be a list. I want to add mac address to this port, but I'm not sure if that's possible or what key I would need to add. Is the structure of the networks object documented somewhere? Thanks --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University

2 3

[ansible] [openstack-cloud] how to find port by IP address in ansible
by jdratlif＠globalnoc.iu.edu 07 Jun '22

07 Jun '22

How can I get a port by it's IP address using the openstack cloud ansible modules? This is what I'm trying, but it never yields any results. I do get results if I use mac_address or device_id until filters, but I need to find the port by it's IP. - name: get port info openstack.cloud.port_info: filters: fixed_ips: - 'ip_address=123.124.125.126' auth: "{{ auth_dict }}" region_name: "{{ region }}" Thanks --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University

1 0

trove, error - Exception during message handling: AttributeError: 'TroveContext' object has no attribute 'notification', please help to fix
by Vasiliy Bragin 07 Jun '22

07 Jun '22

Hello, team trove! trove, error - Exception during message handling: AttributeError: 'TroveContext' object has no attribute 'notification', please help to fix release openstack - yoga, CentOS, CentOS Stream release 8 instance trove - ubuntu, Linux version 5.4.0-113-generic (buildd@lcy02-amd64-067) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 On Yoga Trove guest-agent inside instance not work at all: log 2022-06-01 15:31:13.263 908 DEBUG trove.common.rpc.service [-] Creating RPC server for service guestagent.15e4e2e9-13a4-42f6-bafb-f63fdd074d76 start /opt/guest-agent-venv/lib/python3.8/site-packages/trove/common/rpc/service.py:56 *2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server [-] Exception during message handling: AttributeError: 'TroveContext' object has no attribute 'notification'* 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/osprofiler/profiler.py", line 160, in wrapper 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/trove/guestagent/datastore/manager.py", line 213, in prepare 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server with EndNotification(context, instance_id=CONF.guest_id): 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/trove/common/notification.py", line 48, in __init__ 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server self.context.notification.payload.update(kwargs) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server AttributeError: 'TroveContext' object has no attribute 'notification' 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server Trove guest-agent on instance execute. But DB don't work and not started. After restart guest-agent on instance next log: 2022-06-06 12:51:09.443 4503 DEBUG trove.common.rpc.service [-] Creating RPC server for service guestagent.15e4e2e9-13a4-42f6-bafb-f63fdd074d76 start /opt/guest-agent-venv/lib/python3.8/site-packages/trove/common/rpc/service.py:56 2022-06-06 12:52:09.541 4503 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.update_status run_periodic_tasks /opt/guest-agent-venv/lib/python3.8/site-packages/oslo_service/periodic_task.py:210 *2022-06-06 12:52:09.546 4503 INFO trove.guestagent.datastore.manager [-] Database service is not installed, skip status check* Please tell me how can I fix this errors? -- Vasily Bragin

1 0

[summit][ops][nova] Ops feedback session
by Sylvain Bauza 07 Jun '22

07 Jun '22

Hi all, You probably haven't noticed but we'll run a Forum session at Berlin about Nova pain points and features you'd like to have : https://etherpad.opendev.org/p/nova-berlin-meet-and-greet This will be held tomorrow at 14:50 CEST but your feedback is absolutely welcome even if you can't attend our session. Please take a few minutes to look at the etherpad above and provide your thoughts, that'd be more than welcome. The more we have feedback, the better we could have a productive discussion :-) See you hopefully in-person for whose who are lucky to be present at the Summit. -Sylvain and gibi

1 0

[cinder] Failed to delete volume
by Michal Arbet 07 Jun '22

07 Jun '22

Hi, I would like to ask if someone saw issue when volume failed to delete, log below : This is happening from time to time. 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server [req-80b56352-81b5-47b6-841e-94fface5974e fce19e20993647db8ae39d67094a4294 c8d8a385912c4731832bf1f2cf4727fb - - -] Exception during message handling: EOFError 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 359, in _recv_loop 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server self._read_trampoline() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 331, in _read_trampoline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server timeout_exc=socket_timeout('timed out')) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 210, in _trampoline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server mark_as_closed=self._mark_as_closed) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/hubs/__init__.py", line 159, in trampoline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return hub.switch() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 298, in switch 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.greenlet.switch() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server eventlet.hubs.IOClosed: [Errno 107] Operation on closed file 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/osprofiler/profiler.py", line 160, in wrapper 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/osprofiler/profiler.py", line 160, in wrapper 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "<decorator-gen-754>", line 2, in delete_volume 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/coordination.py", line 144, in _synchronized 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server with lock(blocking): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/locking.py", line 29, in __enter__ 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.lock.__enter__(*self.args, **self.kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/locking.py", line 49, in __enter__ 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server acquired = self.acquire(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/drivers/redis.py", line 58, in wrapper 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/drivers/redis.py", line 110, in acquire 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server blocking=blocking, blocking_timeout=timeout) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/lock.py", line 182, in acquire 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server if self.do_acquire(token): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/lock.py", line 197, in do_acquire 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server if self.redis.set(self.name, token, nx=True, px=timeout): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/client.py", line 1519, in set 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.execute_command('SET', *pieces) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/client.py", line 839, in execute_command 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.parse_response(conn, command_name, **options) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/client.py", line 853, in parse_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server response = connection.read_response() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/sentinel.py", line 55, in read_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return super(SentinelManagedConnection, self).read_response() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 700, in read_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server response = self._parser.read_response() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 310, in read_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server response = self._buffer.readline() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 242, in readline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server self._read_from_socket() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 184, in _read_from_socket 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server data = recv(self._sock, socket_read_size) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/_compat.py", line 71, in recv 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return sock.recv(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 365, in recv 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self._recv_loop(self.fd.recv, b'', bufsize, flags) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 362, in _recv_loop 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server raise EOFError() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server EOFError 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server Thanks, Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Poříčí 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet(a)ultimum.io *https://ultimum.io <https://ultimum.io/>* LinkedIn <https://www.linkedin.com/company/ultimum-technologies> | Twitter <https://twitter.com/ultimumtech> | Facebook <https://www.facebook.com/ultimumtechnologies/timeline>

1 0

[neutron] Bug deputy report for week of May 30th
by Brian Haley 06 Jun '22

06 Jun '22

Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/1976357 - [sqlalchemy-20] Missing DB context in subnetpool methods - https://review.opendev.org/c/openstack/neutron/+/844024 * https://bugs.launchpad.net/neutron/+bug/1976360 - [sqlalchemy-20] Missing DB context in networking-sfc flow classifier - Rodolfo assigned to himself as he's working on a number of these Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/1976292 - [OVN] AgentCache.agents can change during an iteration operation - https://review.opendev.org/c/openstack/neutron/+/843933 * https://bugs.launchpad.net/neutron/+bug/1976323 - [functional/fullstack][master] move fips periodic job to CentOS 9-Stream - https://review.opendev.org/c/openstack/neutron/+/843252 * https://bugs.launchpad.net/neutron/+bug/1976345 - The sync_routers interface takes too long - Needs owner and further investigation * https://bugs.launchpad.net/neutron/+bug/1976366 - OVN: Metadata is not provisioned for a network with DHCP turned off - Related to https://bugs.launchpad.net/neutron/+bug/1918914 ? * https://bugs.launchpad.net/neutron/+bug/1976439 - The database ml2_port_binding_levels and ml2_distributed_port_bindings tables have a lot of redundant data - Needs assignee * https://bugs.launchpad.net/neutron/+bug/1976355 - remove eager subquery load for PortBindingLevel - Recent fix https://review.opendev.org/c/openstack/neutron/+/841823 - https://review.opendev.org/c/openstack/neutron/+/844699 (potential) - Other suggestions from Oleg as well Low bugs -------- * https://bugs.launchpad.net/neutron/+bug/1977629 - [ovn]agent's heartbeat time is error - Zhou Heng took ownership Misc bugs --------- * https://bugs.launchpad.net/neutron/+bug/1977485 - Neutron deletes port in use and nova errors out when cleaning the VM XML - Possible Nova issue, posted comment * https://bugs.launchpad.net/neutron/+bug/1977470 - [networking-bgpvpn] zed unit test failures - Closed as invalid Wishlist bugs ------------- * https://bugs.launchpad.net/neutron/+bug/1976270 - [RFE] Improve performance of bulk port create/delete with networking-generic-switch

1 0

Re: Using infiniband for openstack network communications
by A Monster 06 Jun '22

06 Jun '22

Thank you for your reply, I'll check the link you've sent to me immediately.

3 3

[all][tc] Canceling this week TC meetings
by Ghanshyam Mann 06 Jun '22

06 Jun '22

Hello Everyone, We are cancelling this week's TC meeting and will be meeting next on 16 June. -gmann

1 0

[all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0
by Takashi Kajinami 06 Jun '22

06 Jun '22

Hi, It seems the openstack-tox-docs job is broken in some projects because of the following change made in Sphinx >= 5.0.0 . https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db… We currently have Sphinx 4.5.0 in upper-constraints.txt but some projects (eg. Heat, Nova, Neutron) are using only their own docs-constraints.txt and pull the latest version which is 5.0.1 . IMO we need to fix the problem by the following two steps, but I'd appreciate if anybody can share a better idea. 1. Update tox.ini to honor upper-constraints.txt. This is needed in master and stable/branches as well. 2. Update docs/source/conf.py to hard-code the language parameter to 'en'. This is required in master so that we can bump the upper version later. Thank you, Takashi Kajinami

4 5

Re: Using infiniband for openstack network communications
by A Monster 06 Jun '22

06 Jun '22

Alright, thank you again Sir.

1 0

[kolla] weekly meeting 8th June
by Michał Nasiadka 06 Jun '22

06 Jun '22

Hello koalas, This Wednesday (8th June) meeting is cancelled due to OpenInfra Summit in Berlin. See you on a meeting next week. Michal -- Michał Nasiadka mnasiadka(a)gmail.com

1 0

masakari-hostmonitor error
by Parsa Aminian 06 Jun '22

06 Jun '22

hello on kolla ansible masakari-hostmonitor container on wallaby version I have this error : 2022-06-05 23:30:51.880 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2022-06-05 23:30:51.881 7 ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication is failed. 2022-06-05 23:31:22.042 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Exception caught: Unexpected error while running command. Command: crmadmin -S myhostname Exit code: 102 Stdout: '' Stderr: 'error: Could not connect to controller: Connection refused\nerror: Command failed: Connection refused\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Could you please help me ?

2 1

masakari bug
by Parsa Aminian 05 Jun '22

05 Jun '22

on kolla ansible masakari-hostmonitor container on wallaby version I have this error : 2022-06-05 23:30:51.880 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2022-06-05 23:30:51.881 7 ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication is failed. 2022-06-05 23:31:22.042 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Exception caught: Unexpected error while running command. Command: crmadmin -S myhostname Exit code: 102 Stdout: '' Stderr: 'error: Could not connect to controller: Connection refused\nerror: Command failed: Connection refused\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.

1 0

[openstack-ansible] Meeting on Jun 7th 2022 is cancelled
by Dmitriy Rabotyagov 05 Jun '22

05 Jun '22

Hi everybody! The next meeting on Jun 7th is cancelled due to the Berlin summit. Hope seeing you in person instead! We will have next meeting Jun 14th. -- Kind regards, Dmitriy Rabotyagov

1 0

[tacker] Weekly IRC meeting on Jun 7th cancelled
by Yasufumi Ogawa 05 Jun '22

05 Jun '22

Hi team, The next meeting on Jun 7th is cancelled due to the Berlin summit. Cheers, Yasufumi

1 0

[sdk][openstacksdk][keystone] Does openstacksdk support v3tokenlessauth?
by hai wu 04 Jun '22

04 Jun '22

Does openstacksdk support keystone auth type "v3tokenlessauth"? I got this error message when trying to enable openstacksdk with auth_type="v3tokenlessauth" openstack.exceptions.NotSupported: The identity service for TEST: exists but does not have any supported versions. The following are the possible relevant documents I could find, and there is no place mentioning the support for tokenless auth using openstacksdk: https://docs.openstack.org/openstacksdk/latest/user/config/configuration.ht… https://docs.openstack.org/keystone/train/admin/configure_tokenless_x509.ht…

2 2

[all][tc] What's happening in Technical Committee: summary 03 June 2022: Reading: 5 min
by Ghanshyam Mann 03 Jun '22

03 Jun '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on 02 June. Most of the topics I am summarizing in this email. This was a video call and recordings are available @ https://www.youtube.com/watch?v=nHG2WJNceMs And logs summary are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-06-02-15.02.log.html * We are cancelling next week's meeting. The next TC weekly meeting will be on 16 June Thursday at 15:00 UTC, feel free to add the topic to the agenda[1] by 15 June. 2. What we completed this week: ========================= * TC resolution[2] and documentation[3] for OpenStack release identification schema. This clarify what place we need to use the release number and name. * Security SIG chair rotation from Gage to Jeremy[4] * Removed security-analysis repo from Security SIG[5] * Added a repository for the Large Scale SIG[6] * Added project stats check tool[7] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[8], many items are in-progress. Open Reviews ----------------- * Four open reviews for ongoing activities[9]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian proposal for ""SLURP release notes approach is up for review[10]. Improve project governance --------------------------------- Slawek has the proposal the framework up and it is under review[11]. New ELK service dashboard: e-r service ----------------------------------------------- dpawlik working on ansible playbooks to deploy e-r and functional tests. TripleO team and dpawlik are working on the e-r repo merge. Consistent and Secure Default RBAC ------------------------------------------- We had a call yesterday and discussed about presenting this to the ops meetup in berlin and getting feedback, especially on the 'scope' concept. knikolla plan to moderate this topic in the ops meetup but unfortunately he will not be to make it to the summit. I have added a few notes and what questions we need to ask the operator in etherpad[12]. Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[13]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[14]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[15][16]. Project updates ------------------- * Add a repository for the Large Scale SIG[17] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[18]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [19] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/843214 [3] https://review.opendev.org/c/openstack/governance/+/841800 [4] https://review.opendev.org/c/openstack/governance-sigs/+/844446 [5] https://review.opendev.org/c/openstack/governance/+/844463 [6] https://review.opendev.org/c/openstack/governance/+/843535 [7] https://review.opendev.org/c/openstack/governance/+/810037 [8] https://etherpad.opendev.org/p/tc-zed-tracker [9] https://review.opendev.org/q/projects:openstack/governance+status:open [10] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [11] https://review.opendev.org/c/openstack/governance/+/839880 [12] https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L53 [13] https://review.opendev.org/c/openstack/governance/+/836888 [14] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [15] https://etherpad.opendev.org/p/zuul-config-error-openstack [16] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [17] https://review.opendev.org/c/openstack/project-config/+/843534 [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [19] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

[all][tc] Change OpenStack release naming policy proposal
by Slawek Kaplonski 03 Jun '22

03 Jun '22

Hi, During the last PTG in April 2022 in the TC meeting we were discussing our release naming policy [1]. It seems that choosing appropriate name for every releases is very hard and time consuming. There is many factors which needs to be taken into consideration there like legal but also meaning of the chosen name in many different languages. Finally we decided that now, after Zed release, when we will go all round through alphabet it is very good time to change this policy and use only numeric version with "year"."release in the year". It is proposed in [2]. This is also good timing for such change because in the same release we are going to start our "Tick Tock" release cadence which means that every Tick release will be release with .1 (like 2023.1, 2024.1, etc.) and every Tock release will be one with .2 (2023.2, 2024.2, etc.). [1] https://etherpad.opendev.org/p/tc-zed-ptg#L265[1] [2] https://review.opendev.org/c/openstack/governance/+/839897[2] -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://etherpad.opendev.org/p/tc-zed-ptg#L265 [2] https://review.opendev.org/c/openstack/governance/+/839897

16 42

Fwd: [Forum] Meet the Projects Session!
by Kendall Nelson 03 Jun '22

03 Jun '22

Hello :) I wanted to take a moment to invite project maintainers, core contributors, PTLs, governance officials, etc to a meet the projects (projects being the OIF top level ones- Kata, Zuul, OpenStack, StarlingX, and Airship) session during the Forum at the upcoming Summit in Berlin. It will take place Tue, June 7, 11:20am - 12:30pm local in A - A06. The idea is to gather in a single place so that newer members of the community or people unfamiliar with the project can come and meet some of the active participants and ask questions. It's all really informal so there is no need to prepare anything ahead of time. I realize it is pretty late in everyone's planning their schedules for the summit, but if you have a few minutes to spare to come hang out and meet some people and network, please do! We would love to see you there. -Kendall

5 4

[sdk] openstacksdk 0.99.0 breaks swift object header setting
by Clark Boylan 03 Jun '22

03 Jun '22

Hello, Last week the OpenDev team upgraded Zuul which deployed zuul-executors with openstacksdk==0.99.0 installed. After this, test job logs uploaded to the rackspace swift location were no longer viewable in the Zuul dashboard. The reason for this is these objects did not have properly configured CORS headers any longer. The objects uploaded to OVH appear to have been fine (potentially because the headers are not required for OVH). Today we downgraded the version of openstacksdk on the zuul-executors to 0.61.0 and the rackspace uploaded job log objects have functional CORS headers again. For this reason we're fairly confident that something about the 0.99.0 release is impacting the setting of these headers. The code that does these uploads with openstacksdk can be found here [0]. I suspect that the problem is going to be related to this header set on line 227 [1]. In particular that appears to be rackspace specific and this issue is rackspace specific under 0.99.0. However, I wasn't able to find anything in openstacksdk that would have a problem with this. Does anyone else have ideas? I'm somewhat concerned that this represents a class of data loss (metadata is going missing) for swift object uploads. Additionally, we rely on setting the x-delete-after header to timeout, expire, and prune our logs in swift [2]. If there is a general problem with 0.99.0 setting headers on objects it is possible that all of the objects we created using 0.99.0 do not have this metadata set and will leak in their swift containers. [0] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80a… [1] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80a… [2] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80a…

3 7