June 2022 - openstack-discuss - lists.openstack.org

[all][tc] What's happening in Technical Committee: summary 10 June 2022: Reading: 5 min
by Ghanshyam Mann 11 Jun '22

11 Jun '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We cancelled this week's meeting. The next TC weekly meeting will be on 16 June Thursday at 15:00 UTC, feel free to add the topic to the agenda[1] by 15 June. 2. What we completed this week: ========================= * Added community infrastructure contributor help in upstream investment[2] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[3], many items are in-progress. Open Reviews ----------------- * Five open reviews for ongoing activities[4]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian proposal for ""SLURP release notes approach is up for review[5]. Improve project governance --------------------------------- Slawek has the framework proposal up and is under review[6]. New ELK service dashboard: e-r service ----------------------------------------------- No update this week. Consistent and Secure Default RBAC ------------------------------------------- We called for the operator's feedback on RBAC new directions, I sent an email about it[7] and also created etherpad to collect feedback in a central place[8]. My humble request is to use this central etherpad for feedback/discussion otherwise it might be difficult for me to collect/know feedback conclusions from a lot of 100s places. We have feedback from the berlin summit but we still encourage operators/local events or so keep adding the feedback and your view on RBAC. Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[9]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[10]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[11][12]. Project updates ------------------- * None 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[13]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [14] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/843749 [3] https://etherpad.opendev.org/p/tc-zed-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [6] https://review.opendev.org/c/openstack/governance/+/839880 [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028878.html [8] https://etherpad.opendev.org/p/rbac-operator-feedback [9] https://review.opendev.org/c/openstack/governance/+/836888 [10] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [11] https://etherpad.opendev.org/p/zuul-config-error-openstack [12] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [13] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [14] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Openstack-Ansible vs Kolla-Ansible -Xena or later
by Dave Hall 11 Jun '22

11 Jun '22

Hello, My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am sensitive to the effort that has been put into both of these projects, what I really need right now is the most fool-proof way to deploy and manage a small production cluster for academic instructional use. I have been working for a couple months with OpenStack-Ansible to deploy a small (3-node) test cluster on VirtualBox VMs in preparation for a larger deployment on real hardware - 6 to 10 nodes. (Details: Debian 11 deployment, Debian 10 infra, compute, and storage nodes) It has been slow going, at least partially due to some issues and limitations with VirtualBox. However, deploying a test cluster on VMs still seems preferable to just diving deployment on real hardware and going through multiple scrubs/reinstalls. Today I saw that Mark asked a question about Kolla-Ansible. So, as a 'beginner', should I shift and look at Kolla-Ansible, or should I stay the course and continue with Openstack-Ansible? What are the pros and cons of each? Thanks. -Dave -- Dave Hall Binghamton University kdhall(a)binghamton.edu

6 10

[TripleO] move stable/victoria tripleo to End Of Life OK?
by Marios Andreou 10 Jun '22

10 Jun '22

Hello TripleO the tripleo-ci team proposes that we transition the stable/victoria branch across all tripleo repos [1] to End Of Life [2]. Hopefully this is not a suprise to anyone as it has been discussed for a while (most recent discussion I can point to is at Zed PTG [3]). The stable/victoria branch was moved to Extended Maintenance with [4] so we can no longer make any releases for these repos. Victoria is not an active branch for TripleO (not an FFU branch) so we are just merging patches there on the way back to train. Are there any concerns, objections or any other comment? I'll give this a few days and if there are no further comments I'll post the EOL proposal and update here towards the end of next week, regards, marios [1] https://releases.openstack.org/teams/tripleo.html#victoria [2] https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… [3] https://etherpad.opendev.org/p/tripleo-zed-ci-load [4] https://review.opendev.org/c/openstack/releases/+/837982

1 2

[doc] question/enhancement-request on Sphinx theme used by openstack docs
by Waines, Greg 10 Jun '22

10 Jun '22

Hey ... I am on the technical steering committee for the openstack starlingx team and also provide primary technical review for our docs at https://docs.starlingx.io/ . docs.starlingx.io basically uses the openstack docs team's Sphinx theme for building our html docs ... see https://opendev.org/starlingx/docs . docs.starlingx.io is a very large doc suite. We are interested in improvements to the Table-of-Contents (TOC) navigation and presentation ... but we have limited Sphinx expertise. Specifically we are looking for the TOC support to have - more subsection header levels in TOC // such that user has full view of where he currently is in the doc hierarchy - highlighting to indicate currently selected section - separate scroll bar for TOC // ... so you don't lose sight of selected TOC selection when you scroll down on the main page which has large content. Have you ever considered such doc improvements? Did you have Sphinx expertise to do such improvements ? let us know, Greg.

2 1

[nova] local LVM volume on compute hosts
by Paul Harrison 10 Jun '22

10 Jun '22

I have installed a small test cluster using openstack-ansible (xena) and would like to use local LVM volumes for storing instance volumes - and following https://docs.openstack.org/nova/latest/configuration/config.html and (e.g. https://cloudnull.io/2017/12/nova-lvm-an-iop-love-story/) I have the libvirt section of my /etc/nova/nova.conf as below. -8<--- [libvirt] inject_partition = -2 inject_password = False inject_key = False virt_type = kvm live_migration_with_native_tls = true live_migration_scheme = tls live_migration_inbound_addr = 172.29.236.5 hw_disk_discard = ignore disk_cachemodes = none images_type = lvm images_volume_group = nova volume_clear = zero -8<—— however, the nova service continues to use cinder volumes from my storage host to store instances. Is there something else that needs to be configured to make this work, or is it obsolete?

2 7

[magnum] Docker Swarm errors on Yoga Ubuntu using Install Guide cut-n-paste
by Vince Mulhollon 10 Jun '22

10 Jun '22

I'm trying to use Magnum on a fresh Yoga install on Ubuntu to run some non-load balanced docker swarm workloads. Its a long story but the parts of the swarm workload are heterogeneous so a cluster would just get in the way. Everything else about the cluster is working fine, all is well, etc. I'm a small time VMware and OS admin with years of experience but absolute zero experience with Magnum. Also I have "some" experience with Docker Swarm clusters mostly created and assembled with Ansible. So I know quite well how to look at log files or configs in a general sense, but I have no idea what to expect from Magnum specifically as a "glue" technology. I've followed every detail of the "Magnum Installation Guide" for Yoga as seen at: https://docs.openstack.org/magnum/yoga/install/index.html When provisioning a docker swarm, I very literally cut and paste from: https://docs.openstack.org/magnum/yoga/install/launch-instance.html#provisi… cluster creation fails every time with "Property allowed_cidrs not assigned" exactly as seen on this mailing list post from 2021-August-20 about ten months ago on a Wallaby install, they were also using Ubuntu. http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024273.h… Advice at that point ranged from, try installing Octavia, which I was trying to avoid and I definitely do NOT want a load balancer in front of that particular workload, to Docker swarm is no longer tested for Magnum and it's in practice a K8S exclusive product. I played around for an afternoon and got nowhere WRT configuring the allowed cidrs for a k8s load balancer on a config that does not use a load balancer and does not use k8s. Heat works fine, its super fun, and one alternative is I could give up on Magnum and use some Heat templates, which I know how to do and seems simpler than getting Magnum to work at this point. Or I could brave the scary looking mostly-undocumented world of kuryr and zun, which I was attempting to avoid by using Magnum which is at least simple and easy to install. Or just keep spinning up manual images and letting ansible turn them into docker swarms for me. But, ideally, I'd like to use Magnum to spin up some Docker Swarms for personal curiosity reasons, if nothing else, it would be very convenient if it worked. So the point of this long ramble is, can anyone successfully running Docker Swarm on Yoga on Ubuntu using a very typical "Install Guide" installation provide a pointer or link or github or something that Google searches is not finding to get it to work? Also I hate to piggyback multiple questions but this paragraph also interests me, does anyone have any information and experience with this: https://docs.openstack.org/magnum/yoga/user/index.html#heat-stack-templates "Sample cluster driver To help developers in creating new COE drivers, a minimal cluster driver is provided as an example. The ‘docker’ cluster driver will simply deploy a single VM running Ubuntu with the latest Docker version installed. It is not a true cluster, but the simplicity will help to illustrate the key concepts. To be filled in" That "To be filled in" part would also be very interesting to me as worst case I could shove the Docker Swarm's workload onto a single Docker host, although then I'm back to orchestrating with Heat and presumably Ansible but via a different path.

1 0

[kolla-ansible][xena] Cell database HA/redundancy?
by Norrie, Andrew 10 Jun '22

10 Jun '22

Hi, We are currently planning some large OpenStack deployments utilizing kolla-ansible and I'm curious what folks are doing with the cell database HA/redundancy. With kolla-ansible (xena) it appears that a loadbalancer setup is only allowed for the default database shard (shard 0)... reference: https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb… If we are setting up separate cell database shards with Galera, I'm wondering if there is a convenient work-around or configuration for implementation of HA for these cell databases. In the inventory group_vars directory you can specify the group variables (for each cell database) as: nova_cell_database_address: nova_cell_database_group: but these aren't virtual IP hosts/addresses (non default db shard). This works perfectly fine with a single server cell database but not Galera. If that db server goes down the cell instance information is lost. Many thanks ... Andrew ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately."

3 3

Error creating octavia amphora
by Russell Stather 10 Jun '22

10 Jun '22

Hi I am seeing the below error when creating a load balancer. The security group does exist, and it is the correct group (tagged with charm-octavia) What am I missing here to resolve this? 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', 'message': 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server

4 6

[ops] details for Ops Meetup, Berlin, Tomorrow, Friday 2022-6-10
by Chris Morgan 10 Jun '22

10 Jun '22

A brief twitter thread here https://twitter.com/osopsmeetup/status/1535004027032940544?s=20&t=AgLzV6Qvg… has some reminders about tomorrow's event. Looking forward to seeing everyone. Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[CLOUDKITTY][OpenInfra summit 2022][June 9th] Presentation on Ceilometer, Gnocchi, and CloudKitty
by Rafael Weingärtner 09 Jun '22

09 Jun '22

Hello CloudKitters! It was a great opportunity to be able to present this talk for you all guys. It was a pleasure to see that so many people are interested in CloudKitty, Ceilometer, and Gnocchi. I really hope that we can all collaborate together to keep improving CloudKitty, and other related systems. Attached is the presentation we used during the presentation, so you all can consult this material. Also, as always, if you need some guidance, let us know. Furthermore, as I mentioned in the presentation, I will provide a step-by-step process on how to create a new metric using Ceilometer Dynamic pollster that can collect/generate/derive (however you want to call it) IOPs information for volumes. The new metric name is going to be called "dynamic_pollster.volume.services.volume.iops". Also, the idea is that we collect some attributes, and we use the attribute called "bootable", which indicates if the volume is bootable or not. Then, if the volume is bootable, we will charge for its IOPS; otherwise, we do not charge it. As I mentioned, this is just a very simple use case. It is not targeting anyone's need in production, but it can be used to get you going quite easily with the pipeline. Requirements: - Ceilometer up and running. Version 15.0.0 (or higher is recommended); - Gnocchi up and running. Version 4.3.0 (or higher). If you want to run this in production, this patch might be very important ( https://github.com/gnocchixyz/gnocchi/pull/1059) it has not been merged yet; - CloudKitty up and running. Version 10.0.0 (or higher). With all systems up and running, you will need to do the following: - Ceilometer - Add the new metric collection in the "polling.yaml" file. You can add it in any of the sections you already have there. Or, you can create its own, as follows: ``` - name: volume_iops_metric interval: 600 meters: - dynamic_pollster.volume.services.volume.iops ``` - After declaring the metrics there, you can create the pollster in the "pollster.d" folder in Ceilometer. The name of the file can be anything, as long as it ends with ".yaml"; for instance, "pollsters.d/volume-iops.yaml". Then, you will need to add the following content. Remember to replace the contents of "<>" with the values of your environment. ``` - name: "dynamic_pollster.volume.services.volume.iops" response_entries_key: "volumes" next_sample_url_attribute: "volumes_links | filter(lambda v: v.get('rel') == 'next', value) | list(value) | value[0] | value.get('href')" sample_type: "gauge" unit: "GB" value_attribute: "volume_type" endpoint_type: "volume" url_path: "/v3/<SERVICE_UUID_FOR_CEILOMETER_USER>/volumes/detail?all_tenants=true&limit=10" headers: "Openstack-API-Version": "latest" project_id_attribute: "os-vol-tenant-attr:tenant_id" metadata_fields: - "status" - "name" - "volume_type" - "availability_zone" - "user_id" - "bootable | str(value)" - "attachments | value or [ { 'server_id': '' } ] | value[0] | value['server_id']" metadata_mapping: "availability_zone": "dynamic_availability_zone" "name": "dynamic_display_name" "volume_type": "dynamic_volume_type" "attachments | value or [{ 'server_id': '' }] | value[0] | value['server_id']": "dynamic_instance_id" "bootable | str(value)": "bootable" value_mapping: "<VOLUME_TYPE_UUID_OR_VOLUME_TYPE_NAME>": "<THE IOPS THAT THIS VOLUME TYPE REPRESENTS>" ``` - After creating that, you will need to customize the file "gnocchi_resources.yaml", which is the file that maps metrics to resources, and attributes that are pushed to the backend. You can also create your own "gnocchi_resources.yaml" file, and then use it in the pipeline configuration in the "pipeline.yaml" file, in the configuration of "publishers". Then, you would have something like "gnocchi://?resources_definition_file=%2Fetc%2Fceilometer%2Fcustom_gnocchi_resources.yaml", assuming the file is called "custom_gnocchi_resources.yaml". In this file, you will need to add the new metric and attribute to be pushed to the metrics backend. The following is an example of "volume" resource declaration in the "gnocchi_resource.yaml" file with the new metric and attribute that we want to capture. ``` - resource_type: volume metrics: <MANY OTHER METRICS HERE> dynamic_pollster.volume.services.volume.iops: attributes: display_name: resource_metadata.(dynamic_display_name|notification_display_name) bootable: resource_metadata.bootable <MANY OTHER ENTRIES HERE> ``` - After the above configuration, you will need to create the new attribute in the Gnocchi resource type. To achieve that, you can issue the following command. ``` gnocchi resource-type update -a bootable:string:false volume ``` - Then, you will need to restart the Ceilometer. - Now, you can issue "gnocchi resource show <VOLUME ID>", and you should see the new metric in the resource, and the new attribute as well. With Ceilometer and Gnocchi configured for the new metric, we can move on to CloudKitty. - The new metric needs to be declared in the "metrics.yml" file. The content should be something similar to the following (adapt as you need). ``` dynamic_pollster.volume.services.volume.iops: unit: IOPS alt_name: disk-volume-iops-usage-hours groupby: - id - user_id - project_id - volume_type - display_name - revision_start - instance_id - availability_zone - bootable metadata: [] extra_args: aggregation_method: max resource_type: volume use_all_resource_revisions: false ``` - Restart CloudKitty processor - With the new metric introduced in CloudKitty processor, you will need to create the rating configuration. In our case, we wanted to rate the volumes marked as bootable for IOPs. The volumes that are not bootable would not be rated. The following are the steps one needs to execute to create the hashmap rating rules. - Create the service ``` openstack rating hashmap service create disk-volume-iops-usage-hours ``` - Then, create the field ``` openstack rating hashmap field create <SERVICE_ID> <FIELD NAME WE WANT TO RATE, IN OUR CASE, IT IS bootable> ``` - After that, we can create the rating rule ``` openstack rating hashmap mapping create --field <FIELD UUID THAT WE CREATE> --value 'true' 1 --type flat ``` - Having done that, you should be able to rate all volumes by IOPS if they have the flag "bootable" set to true. Otherwise, they will not be rated. That is all that one needs to execute to replicate the example I showed during the presentation. If you guys have any doubts or difficulties when using CloudKitty, do not hesitate to ping me. -- Rafael Weingärtner

1 0