April 2022 - openstack-discuss - lists.openstack.org

[neutron] Bug Deputy Report January 03 - 10
by Lucas Alvares Gomes 11 Apr '22

11 Apr '22

Hi, This is the Neutron bug report from January 3rd to 10th. Critical: * https://bugs.launchpad.net/neutron/+bug/1956344 - "Functional test test_gateway_chassis_rebalance is failing intermittently" - Assigned to: Rodolfo Alonso High: * https://bugs.launchpad.net/neutron/+bug/1956034 - "ovn load balancer health monitor cause mac address conflict" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1956745 - " [ovn-octavia-provider] Load Balancer remained with ACTIVE state even with PENDING_UPDATE listener" - Assigned to: Luis Tomas Bolivar * https://bugs.launchpad.net/neutron/+bug/1956763 - "[OVN] Overlapping configuration for security group logging is not applied correctly" - Assigned to: Elvira Garcia Ruiz Medium: * https://bugs.launchpad.net/neutron/+bug/1956035 - "ovn load balancer member failover not working when accessed from floating ip" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1956632 - "DNS integration documentation and scenario tests need update after dns_assignment calculation was changed" - Assigned to: Miguel Lavalle * https://bugs.launchpad.net/neutron/+bug/1956785 - "Duplicate validator TypeError for dict values Edit" - Assigned to: Harald Jensås Low: * https://bugs.launchpad.net/neutron/+bug/1956476 - "[OVN] Disallow multiple physnets per bridge" - Assigned to: Rodolfo Alonso * https://bugs.launchpad.net/neutron/+bug/1956770 - " keepalived no_track implemented in >= 2.0.3" - Assigned to: Rodolfo Alonso Needs further triage: * https://bugs.launchpad.net/neutron/+bug/1956435 - "OVS: support multiple segments per host" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1956460 - "The VM can't get metadata from dhcp-agent" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1956846 - "ha router duplicated routes" - Unassigned Cheers, Lucas

4 7

[zuul][neutron][neutron-fwaas][tap-as-a-service] Errors in the jobs definitions and EOL of some old neutron-fwaas and tap-as-a-service branches
by Lajos Katona 11 Apr '22

11 Apr '22

Hi, During the PTG we got questions about projects which cause zuul config errors (see [1]). Slawek started a thread regarding networking-midonet (see [2]), I join to that with neutron-fwaas and tap-as-a-servie. - neutron-fwaas' stable/rocky and stable/queens branches - tap-as-a-service stable/ocata, stable/pike, stable/queens and stable/rocky branches are affected - as tap-as-a-service was under x/ namespace previously I am not sure what can happen to those branches. If there is nobody to fix the config and gate errors on these old branches, then the best would be to EOL and later delete these branches. [1]: https://zuul.opendev.org/t/openstack/config-errors [2]: http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028120.ht…

1 0

[requirements][kolla][infra] Python versions in u-c
by Marcin Juszkiewicz 11 Apr '22

11 Apr '22

OpenStack Zed lists Python 3.8 as minimal version. Debian 'bullseye' (current stable) as Python 3.9, Ubuntu 20.04 'focal' has 3.8. Ubuntu 22.04 'jammy' has 3.10, CentOS Stream 9 (so RHEL 9, Rocky Linux 9) has 3.9 version. We ignore anything RHEL 8 based as they have Python 3.6 only and there is no work on getting OpenStack Zed supported there. So there will be 3.8, 3.9, 3.10 used with Zed. Two question to requirements team: 1. Are there plans to get rid of 3.6 entries from openstack/requirements/upper-constraints.txt in near future? 2. Can we get "python_version=='3.8'" lines be changed to ">=" ones? Or once 3.6 entries drop just assume that we have 3.8+ and drop 'python_version' entries? websocket-client===1.3.1;python_version=='3.6' websocket-client===1.3.2;python_version=='3.8' Asking due to my recent Kolla/infra work on getting CentOS Stream 9 supported in both places. Infra team has CI job which builds Python wheel cache to make sure that other CI jobs do not have to do it. Times are nicely cut due to this stuff. There are 55 entries with "python_version=='3.8'". Probably none of them are time critical anymore as upstream projects provide aarch64 wheels most of times too.

2 1

[Zuul] [neutron] Errors in the jobs definitions and EOL of some old networking-midonet branches
by Slawek Kaplonski 11 Apr '22

11 Apr '22

Hi, Today I got back to checking my old patches which are going to fix some zuul jobs' definitions. And I found that CI for networking-midonet in some stable branches is totally broken: * stable/ussuri: https://review.opendev.org/c/openstack/networking-midonet/+/823273[1] - everything is red here * stable/train: https://review.opendev.org/c/openstack/networking-midonet/+/823275[2] - most of the jobs are red * stable/stein: https://review.opendev.org/c/openstack/networking-midonet/+/823276[3] - here there were errors in the definition of the jobs, I changed it now, maybe it will run, So my question is if we want, and have resources, to fix it somehow or should we maybe set those branches as EOL now? Also, another question to the Zuul team - will it be ok if such branch will be EOL to get rid of the Zuul configuration errors warnings? Or should we somehow force to merge those patches first to get rid of the errors in https://zuul.opendev.org/t/openstack/config-errors[4] ? -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://review.opendev.org/c/openstack/networking-midonet/+/823273 [2] https://review.opendev.org/c/openstack/networking-midonet/+/823275 [3] https://review.opendev.org/c/openstack/networking-midonet/+/823276 [4] https://zuul.opendev.org/t/openstack/config-errors

2 2

[all][neutron] Pike - End of Life
by Lajos Katona 11 Apr '22

11 Apr '22

Hi, As time and cycles are passing it's time to bring up the topic of EOLing an old branch again. In Neutron (and its satellite projects, aka stadium) stable/pike has had low activity in the last cycle and recently even the jobs for it started to fail, and we cause some zuul config errors which are hard to fix as the gate is broken. Based on the Ocata EOL mail (see [1]), Neutron team decided to EOL Pike, see the discussions during the Zed PTG (see [2]) [1]: http://lists.openstack.org/pipermail/openstack-discuss/2021-April/021949.ht… [2]: https://etherpad.opendev.org/p/neutron-zed-ptg#L254 Lajos Katona (lajoskatona)

1 0

[vmware-nsx] EOL old branches
by Slawek Kaplonski 11 Apr '22

11 Apr '22

Hi Salvatore, According to Your comments in my patches to x/vmware-nsx project: * stable/stein: https://review.opendev.org/c/x/vmware-nsx/+/823277[1] * stable/rocky: https://review.opendev.org/c/x/vmware-nsx/+/823291[2] Can You maybe make them EOL finally? -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://review.opendev.org/c/x/vmware-nsx/+/823277 [2] https://review.opendev.org/c/x/vmware-nsx/+/823291

1 0

[neutron] Bug Deputy Report April 04 - 11
by Lucas Alvares Gomes 11 Apr '22

11 Apr '22

Hi, This is the Neutron bug report from April 4th to 11th. Critical: * https://bugs.launchpad.net/neutron/+bug/1967893 - " [stable/yoga] tempest.scenario.test_network_qos_placement.MinBwAllocationPlacementTest fails in neutron-ovs-tempest-multinode-full job Edit" - Assigned to: Lajos Katona High: * https://bugs.launchpad.net/neutron/+bug/1967839 - " [L3] NDP extension not handing "ha_state_change" correctly" - Assigned to: Rodolfo Alonso * https://bugs.launchpad.net/neutron/+bug/1967996 - "[OVN] External subscribers to feed generated in Openstack fails if no internal VM is subscribed to that feed first" - Unassigned Medium: * https://bugs.launchpad.net/neutron/+bug/1967742 - "ML2 - Network Context, not possible to see original/current segments" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1968343 - "Security Group Rule create with forged integer security_group_id causes exceptions" - Assigned to: Andrew Karpow Needs further triage: * https://bugs.launchpad.net/neutron/+bug/1968057 - "Doc needed for configuring neutron AZs for edge" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1968269 - " [router qos] qos binding is not clear after remove gateway" - Unassigned Cheers, Lucas

1 0

[Senlin][Zuul] Errors in the jobs definitions in stable branches
by Slawek Kaplonski 11 Apr '22

11 Apr '22

Hi, Some time ago I pushed patches to fix definitions of the Zaqar's jobs in stable branches: * stable/stein: https://review.opendev.org/c/openstack/senlin/+/823287[1] * stable/rocky: https://review.opendev.org/c/openstack/senlin/+/823293[2] * stable/queens: https://review.opendev.org/c/openstack/senlin/+/823294[3] It seems that CI for all those branches is broken. Do You still have plans to fix it so I can rebase my patches and we can get rid of those zuul's configuration errors? Or should those branches be maybe EOL? -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://review.opendev.org/c/openstack/senlin/+/823287 [2] https://review.opendev.org/c/openstack/senlin/+/823293 [3] https://review.opendev.org/c/openstack/senlin/+/823294

1 0

[Zaqar][Zuul][TC] Errors in the jobs definitions in stable branches
by Slawek Kaplonski 11 Apr '22

11 Apr '22

Hi, Some time ago I pushed patches to fix definitions of the Zaqar's jobs in stable branches: * stable/stein: https://review.opendev.org/c/openstack/zaqar/+/823289[1] * stable/rocky: https://review.opendev.org/c/openstack/zaqar/+/823295[2] * stable/queens: https://review.opendev.org/c/openstack/zaqar/+/823296[3] * stable/pike: https://review.opendev.org/c/openstack/zaqar/+/823297[4] It seems that CI for all those branches is broken. Do You still have plans to fix it so I can rebase my patches and we can get rid of those zuul's configuration errors? Or should those branches be maybe EOL? I'm tagging TC in the topic of that email too as there is still no PTL of the Zaqar project appointed. -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://review.opendev.org/c/openstack/zaqar/+/823289 [2] https://review.opendev.org/c/openstack/zaqar/+/823295 [3] https://review.opendev.org/c/openstack/zaqar/+/823296 [4] https://review.opendev.org/c/openstack/zaqar/+/823297

1 0

error
by Reza Bojnordi 11 Apr '22

11 Apr '22

Hi, sorry I have problem on Openstack tcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw' Apr 09 20:09:34 infra1-nova-api-container-483c43d7 nova-metadata-wsgi[2912]: 2022-04-09 20:09:34.836 2912 ERROR stevedore.extension [req-089ca62f-02ee-49f9-a305-75f426102ebc - - - - -] Could not load 'oslo_cache.e tcd3gw': No module named 'etcd3gw': ModuleNotFoundError: No module named 'etcd3gw' -- <https://about.me/rbojnordi?promo=email_sig&utm_source=product&utm_medium=em…> reza bojnordi about.me/rbojnordi <https://about.me/rbojnordi?promo=email_sig&utm_source=product&utm_medium=em…>

4 4

[baremetal-sig][ironic] Tue Apr 12, 2022, 2pm UTC: "Using OpenStack Ironic for HPC at Berlin Institute of Health"
by Arne Wiebalck 11 Apr '22

11 Apr '22

Dear all, The Bare Metal SIG will meet tomorrow Tue Apr 12, 2022, 2pm UTC featuring a topic of the day presentation by Dr. Manuel Holtgrewe: "Bare Metal for Health - Using OpenStack Ironic for HPC at Berlin Institute of Health" Everyone is welcome, all details on how to join can be found on the SIG's etherpad: https://etherpad.opendev.org/p/bare-metal-sig Hope to see you there! Arne -- Arne Wiebalck CERN IT

1 0

[Glance] Zed PTG Summary
by Abhishek Kekane 11 Apr '22

11 Apr '22

Hi All, We had our fifth virtual PTG between 4th April to 8th April 2022. Thanks to everyone who joined the virtual PTG sessions. Using bluejeans app we had lots of discussion around different topics for glance, glance + cinder, fips and Secure RBAC. I have created etherpad [1] with Notes from the session and which also includes the recordings of each discussion. Here is a short summary of the discussions. Tuesday, April 5th 2022 # Yoga Retrospective On the positive note, we managed to complete all the work we targeted for the yoga cycle. In addition to that we have organized a first ever glance review party where we managed to perform group reviews which helped us to cover our review load in the final milestone. On the other side we need to reorganize our bi-weekly bugs meeting and also improve our documentation and API references. Recording: https://bluejeans.com/s/kVeDPbu6kX2 - Chapter 1 # Cache API - New API to trigger periodic job In Yoga we have managed to put together new endpoints for cache, this cycle we should add a new API to trigger the periodic job to cache the images. Here we have decided to get rid of periodic job and add a new API to cache(pre-cache) the specified image(s) instantly rather than waiting for the next periodic run to pre-cache those images. Recordings: https://bluejeans.com/s/kVeDPbu6kX2 - Chapter 2 # Glance Cache improvements, restrict duplicate downloads How we can avoid multiple downloading of the same image in cache on first download. Final design - https://review.opendev.org/c/openstack/glance-specs/+/734683 Recordings: https://bluejeans.com/s/kVeDPbu6kX2 - Chapter 3 # Distributed responsibilities among cores/team >From this cycle we have decided to follow a distributed leadership model internally which will help us to train internal members to take the PTL responsibilities in the upcoming cycle. We have decided to distribute the below responsibilities among ourselves in this cycle. Release management: pranali/abhishek Bug management: cyril/abhishek Meetings:pranali Stable branch management: jokke Cross project communication:abhishekk Mailing lists:pranali/abhishekk PTG/summit preparation:pranali/abhishekk Vulnerability management: jokke Infra management: abhishekk Recordings: https://bluejeans.com/s/kVeDPbu6kX2 - Chapter 4 # Secure RBAC - System Scope Unfortunately due to time crunch we were not able to find answers to a few of our queries in this PTG, so we have decided to attend the Open office hours and Policy popup meetings to get them sorted. As per community goal we should be enforcing the new RBAC policies from this cycle and support system-admin. Once we get our doubts sorted then we will share more information about the same. Recordings: https://bluejeans.com/s/kVeDPbu6kX2 - Chapter 5 # Policy refactoring - Part 2 In Xena we have managed to move all policy checks to the API layer. This cycle we need to work on removing dead code of policy and authorization layer. So we are going to ensure that the policy and authorization layer are not used anywhere before removing it from the code base. Recordings: https://bluejeans.com/s/kVeDPbu6kX2 - Chapter 6 Wednesday, April 06th 2022 # Proposal for moving away from onion architecture This is a long term goal and from this cycle we should start doing homework on how we can squash two or more layers together and move away from the onion architecture and obtain a popular and simple MVC architecture in upcoming cycles. This cycle we will be mostly working on finalizing the detail spec about this work. Recordings: https://bluejeans.com/s/ZmURX2kOeJy - Chapter 1 # Import image from another region When dealing with a multi-region cloud it often appears that operators or customers need to copy images from a region to another (it can be public images or even remote backup of instance snapshot). It is currently complicated to implement as customers need to save the image locally, then upload it to the new region. We propose to rely mostly on the web-download code of glance to directly download an image from a remote glance, calling this method « glance-download ». Note that this first version will require a federated Keystone between all the glance in order to avoid all authentication problems (we will rely on the context token of the target glance to authenticate to the remote glance). Recordings: https://bluejeans.com/s/ZmURX2kOeJy - Chapter 2 # Expanding stores-info detail for other stores In Yoga we added a new API ``stores-detail`` to expose the properties of stores but currently it is only exposing details of rbd store, we are planning to extend its support to expose properties of other stores as well. Recordings: https://bluejeans.com/s/ZmURX2kOeJy - Chapter 3 # Discussion of property injection coherency between image import and possible implementation in upload Glance does support injecting certain properties for images created by non-admin users by using inject metadata import plugin, but same is not possible when we do not use import workflow and use traditional way to create the image. This cycle we will be working on supporting injecting the metadata while creating images using upload workflow. https://bluejeans.com/s/ZmURX2kOeJy - Chapter 4 Thursday, April 07th 2022 # Refactor glance cinder store Currently we have one file for cinder store where the logic for handling all cinder backend types (on basis of protocol) exists. It makes the code less readable and prone to mistakes and even the unit testing for code coverage is difficult due to a lot of nested code. The idea of this proposal is to divide the one big file into backend specific files on the basis of operations sharing the same interface. Eg: connect_volume call for most of the backends is the same except for remotefs type drivers where it is handled in a custom way. https://bluejeans.com/s/oiS49m8Pj_o - Chapter 1 # Native Image Encryption Barbican updates: - Microversion is done - Secret consumer will be implemented by milestone 1 - Luzi is interested to work on image encryption - Glance will keep watch and review the work if posted this cycle # Default Glance to configure multiple stores Glance has deprecated single stores configuration since Stein cycle and now will start putting efforts to deploy glance using multistore by-default and then remove single store support from glance. This might be going to take a couple of cycles, so in yoga we are going to migrate internal unit and functional tests to use multistore config and also going to modify devstack to deploy glance using multistore configuration for swift and Ceph (for file and cinder it's already supported). We need to notify respected deployment teams (ansible/tripleo/ceph-admin) about our work and we are moving from single store configuration. Recordings: https://bluejeans.com/s/oiS49m8Pj_o - Chapter 2 # Fips overview Path forward: - add experimental/periodic fips job on centos 8 (it will run on master) - centos 9 dependency fixes (tempest and swift changes) - Once dependency merges, experimental/periodic job will be run for centos 9 (enough time to verify that it is stable now) - Once it is stable, move it to check/gate queue - Then backport swift/tempest dependencies to stable branches - Run centos 9 fips job as periodic on stable branches - Once stable move those to gate/check queue for stable branches Recordings: https://bluejeans.com/s/oiS49m8Pj_o - Chapter 3 # Cross project meet with Cinder Discussion 1: New API to expose location information We have OSSN-0065 describing the security risk of enabling ``show_multiple_locations`` option but this is required for cinder to perform certain optimizations when creating a volume from image (in case of cinder and RBD store). The proposal is to create a new admin only API to provide the location of the image and avoid dependency on the config options. Decided to write a spec describing the current API design for the new locations API (alternative: nova's approach of using alternative endpoint and service role/token as well) Discussion 2: Clone v2: RBD deferred deletion Recently cinder has utilized Ceph clone v2 support for its RBD backend, since then if you attempt to delete an image from glance that has a dependent volume, all future uses of that image will fail in error state. Despite the fact that the image itself is still inside of Ceph/Glance. This issue is reproducible if you are using ceph client version greater than 'luminous'. Decided to fix things on cinder side and see how we can fix glance using the same techniques (also document it since customers face these issues all the time) Recordings: https://bluejeans.com/s/efAqf0e5RDQ - Chapter 2 Friday, April 08th 2022 # Image Export with metadata Especially if we implement the glance-download discussed on Wednesday it might be worth exploring my old idea of image export, which would bundle the image metadata together with the image payload itself for easier import into the glance environment. This will need bits of work on both sides of the process. The source will need to be able embed the metadata into the end of the datastream sent to the client and the receiving end will need to understand how to pick up and parse that data. To make easier transfer of the image (especially RAW images from all of our Ceph deployments) the original image payload should be compressed on the fly when sent to the client and the metadata can be added after the compression is closed. This way the image can be brought in older glance deployments too that supports the uncompression and if it doesn't know to look at the metadata that part will be just simply ignored. Recordings: https://bluejeans.com/s/C7F5pDPR_v4 - Chapter 1 You will find the detailed information about the same in the PTG etherpad [1] along with the recordings of the sessions and milestone wise priorities at the bottom of the etherpad. Kindly let me know if you have any questions about the same. [1] https://etherpad.opendev.org/p/zed-glance-ptg Thank you, Abhishek

1 0

Xena and CEPH RBD backend (show_image_direct_url status )
by west, andrew 10 Apr '22

10 Apr '22

Hello experts Currently using openstack Xena and Ceph backend (Pacific 16.2.7) It seems there is a bug (since Wallaby?) where the efficient use of a CEPH Pacific RBD backend (i.e with copy-on-write-cloning) is not working . Show_image_direct_url needs to be False to create volumes (or ephemeral volumes for nova) This can of course be tremendously slow (Nova , ephemeral root disk) without copy-on-write cloning feature of Ceph. As Ceph RBD is THE most favourite backend for block storage in openstack I am wondering how others are coping (or workarounds found ?) Which combinations of Openstack and Ceph are known to work well with copy-on-write-cloning? How is the noted GRAVE Security RISK of enabling Show_image_direct_url mitigated ? (i.e I think , for CEPH RBD, it needs to be True to get cloning to work efficiently) See another report of this issue here: Re: Ceph Pacifif and Openstack Wallaby - ERROR cinder.scheduler.flows.create_volume - CEPH Filesystem Users (spinics.net)<https://www.spinics.net/lists/ceph-users/msg66016.html> Thanks for any help or pointers, Andrew West Openstack consulting CGG France ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately."

7 9

Magnum not working / kube minions not spawning
by Oliver Weinmann 09 Apr '22

09 Apr '22

Hi, I recently deployed Openstack wallaby using kolla-ansible and also deployed magnum. I know it was working fine a while ago and I was able to spin up K8s clusters without a problem. But I think This was on Ussuri back then. I went through the magnum troubleshooting guide but couldn’t solve my problem. Magnum spins up the master node and I can log in via SSH using its floating IP. I checked the logs and saw this after waiting for a few minutes: role.kubernetes.io/master="" + echo 'Trying to label master node with node-role.kubernetes.io/master=""' + sleep 5s ++ kubectl get --raw=/healthz Error from server (InternalError): an error on the server ("[+]ping ok\n[+]log ok\n[+]etcd ok\n[+]poststarthook/start-kube-apiserver-admission-initializer ok\n[+]poststarthook/generic-apiserver-start-informers ok\n[+]poststarthook/start-apiextensions-informers ok\n[+]poststarthook/start-apiextensions-controllers ok\n[-]poststarthook/crd-informer-synced failed: reason withheld\n[-]poststarthook/bootstrap-controller failed: reason withheld\n[-]poststarthook/rbac/bootstrap-roles failed: reason withheld\n[-]poststarthook/scheduling/bootstrap-system-priority-classes failed: reason withheld\n[+]poststarthook/apiserver/bootstrap-system-flowcontrol-configuration ok\n[+]poststarthook/start-cluster-authentication-info-controller ok\n[+]poststarthook/start-kube-aggregator-informers ok\n[+]poststarthook/apiservice-registration-controller ok\n[+]poststarthook/apiservice-status-available-controller ok\n[+]poststarthook/kube-apiserver-autoregistration ok\n[-]autoregister-completion failed: reason withheld\n[+]poststarthook/apiservice-openapi-controller ok\nhealthz check failed") has prevented the request from succeeding + '[' ok = '' ']' + echo 'Trying to label master node with node-role.kubernetes.io/master=""' + sleep 5s Trying to label master node with node-role.kubernetes.io/master="" ++ kubectl get --raw=/healthz + '[' ok = ok ']' + kubectl patch node k8s-test-small-cal-zwe5xmigugwj-master-0 --patch '{"metadata": {"labels": {"node-role.kubernetes.io/master": ""}}}' Error from server (NotFound): nodes "k8s-test-small-cal-zwe5xmigugwj-master-0" not found + echo 'Trying to label master node with node-role.kubernetes.io/master=""' Running kubectl get nodes is just empty even when appending all-namespaces. I pretty much used the documentation that I created when I was using Ussuri. I wonder what has changed since then that would make this fail. I googled for hours but was not able to find similar issues and if then it was about having different version of k8s server and client. Which is definitely not the case. I also tried this on Xena but it also fails. I do have the feeling that the issue is network related but I do not see any issues at all spinning up instances and also the communication between instances works fine. Here are my current configs: Globals.yml [vagrant@seed ~]$ grep ^[^#] /etc/kolla/globals.yml --- kolla_base_distro: "centos" kolla_install_type: "source" openstack_release: "wallaby" kolla_internal_vip_address: "192.168.45.222" kolla_external_vip_address: "192.168.2.222" network_interface: "eth2" neutron_external_interface: "eth1" keepalived_virtual_router_id: "222" enable_haproxy: "yes" enable_magnum: “yes” multinode hosts file control[01:03] ansible_user=vagrant ansible_password=vagrant ansible_become=true api_interface=eth3 compute[01:02] ansible_user=vagrant ansible_password=vagrant ansible_become=true api_interface=eth3 [control] # These hostname must be resolvable from your deployment host control[01:03] # The above can also be specified as follows: #control[01:03] ansible_user=vagrant ansible_password=vagrant ansible_become=true #compute[01:02] ansible_user=vagrant ansible_password=vagrant ansible_become=true # The network nodes are where your l3-agent and loadbalancers will run # This can be the same as a host in the control group [network] control[01:03] #network01 #network02 [compute] compute[01:02] [monitoring] control[01:03] #monitoring01 # When compute nodes and control nodes use different interfaces, # you need to comment out "api_interface" and other interfaces from the globals.yml # and specify like below: #compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1 [storage] control[01:03] #storage01 [deployment] localhost ansible_connection=local cat /etc/kolla/config/magnum.conf [trust] cluster_user_trust = True Sorry for the formatting. Sending this on a smartphone with plenty of copy and paste. Best Regards, Oliver

1 1

[tc][all][ Technical Committee Zed Virtual PTG discussions details
by Ghanshyam Mann 09 Apr '22

09 Apr '22

Hello Everyone, I am writing the Technical Committee discussion details from the Zed cycle PTG this week. We had a lot of discussions and that is why it is a long email but I tried to categorize the details per topic so that you can easily spot the discussion you are interested in. TC + Community leader's interaction ============================ We continue the interaction session in this PTG also. The main idea here is to interact with community leaders and ask for their feedback on TC. I am happy to see more attendance this time. Below are the topics we discussed in this feedback session. * Updates from TC: ** Status on previous PTG interaction sessions' Action items/feedback: *** TC to do more technical guides for the project: DONE We have a unified limit as the first technical guide to start in this[1]. *** TC to define a concrete checklist to check goal readiness before we select a goal: DONE We decoupled the community-wide goal from the release cycle[2] and also added the goal readiness checklist[3]. *** Spread the word: DONE We have updated a new page in project-team-guide where projects can spread the word/story[4] ** Do not do ignorant Recheck: Recheck without seeing the failure even those are not related to the proposed change is very costly and end up consuming a lot of infra resources. Checking the failure and adding comments in why you are doing a recheck will give related projects a chance to know the frequent failures happening in upstream CI and so does we can spend more time fixing them. Dan Smith has added detailed documentation on "How to Handle Test Failures"[5], this can be used to educate the contributors on recheck and how to debug the failure reason. *** Action Item: **** PTL to start spreading the word and monitor ignorant recheck in their weekly meeting etc. **** On ignorant recheck, Zuul to add a comment with the link to recheck practice documentation. ** Spread the word/project outreach: This was brought up again in this PTG but I forget to update that we did work on this in the Yoga cycle and updated the project-team-guide to list the places where the project can spread the word/story[4]. It is ready for projects to refer it and know where they can post their stories/updates. * Recognition to the new contributors and encourage/help them to continue contributing to OpenStack: When there is a new contributor who contributes a significant amount of work to any project then we should have some way to appreciate them. It can be an appreciation certificate/badge from the foundation or TC, appreciation on social media, especially on linked-in. In TC, we will check what is the best possible and less costly way to do so. We checked in cross-community session with k8s and they seem to have coupon/badge distribution from CNCF foundation. 2021 user survey analysis =================== Jay has prepared the summary of the 2021 TC survey[6] and we covered it at a high level, especially the big changes from the previous survey. As the next step, we will review the proposed summary and merge it. Also, we talked about updating the upgrade question when we start the tick/tock release. Prepare to migrate for SQLAlchemy 2.0 ============================== sqlalchemy 2.0 is under development and might have many incompatible changes. To prepare OpenStack in advance, Stephen sent the mail on what are changes and project needs to do[7], also gave the project status. oslo.db is all good to use the sqlalchemy 2.0 and the required neutron change is also merged. Thanks to Stephen for driving it and taking care of various project work. Elections Analysis: ============== Project elections are not going well and we end up with a lot of missing nominations. This cycle, we had 17 projects on the leaderless list and out of it, 16 missed the nomination. There are various factors that lead to missing nomination, short notice from the election (1-2 weeks), PTLs are not active on ML, Language, Time Zone etc. By seeing the number of leaderless projects end up having the new PTLs (this cycle it was just 1 project) and having such projects repeating the nomination miss, we thought election in those projects are not actually required. TC agrees to change the leader assignment for such a project and instead of finding PTL, we will propose to move these projects to the DPL model. WIth the DPL model, we do not need to repeat the PTL assignment in every cycle. Please note, moving them to DPL model will not be automatically done instead that will be done when there are required liaisons to take care of DPL responsibility. Community-wide goals Check In & discussion: =================================== * FIPS goal ade_lee gave the current status of this FIPS work progress in various projects[8]. Current FIPS jobs are based on the centos9 stream which is not so stable but we will keep monitoring it. Canonical has suggested a way to set up FIPS jobs on ubuntu and will give the foundation keys to do FIPS testing and periodically rotate them. One challenge is the horizon which has the dependencies on Django fix and that will be present in Django higher version (4.0 or later) and it is difficult for the horizon to upgrade to that, especially the current bandwidth team has. He also proposed the different milestones to finish the FIPS work[9]. The Technical Committee agreed to have this as a 'selected goal' and as the next step ade_lee will propose it in governance with defined milestone and we will review it. Thanks, ade_lee for all your work and energy in this. * RBAC goal I have summarised it in a separate email, please refer to it - http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028103.ht… Release Cadence Adjustment: ======================= In TC and leaders' interaction sessions, Dan presented an overview of the new release cadence and explain the things which will be changed. After that project followed the discussion in their project PTG sessions and then brought the question in the Thursday TC session. There was a question if we can remove the things (which are deprecated in tick release) in tock release and the answer is yes. We discussed the automation of deprecation/upgrade release notes from tock release to tick release and it is fine but we need to make sure we do not make tick release notes so big that people start ignoring those. We will see how it goes in a few of the initial tick release notes. Below are the various other things which need changes: * ACTION: ** Deprecation document in project-team-guide[10]: *** Add the master deployment case as a recommendation *** Intermediate release upgrade case *** Update 4-a paragraph to clarify the 12 months case and things deprecated in tick can be removed in the next tock release. ** Testing runtime *** In tock release, it is ok to add/test the new python version but do not upgrade the distro version. *** In tick release: no change and keep doing the same what we do currently ** Stable branch *** Update release and stable team with what is proposed in https://governance.openstack.org/tc/resolutions/20220210-release-cadence-ad… ** tick-tock name: ***gmann to check foundation about trademark check on "tick", "tock" name Remove release naming instead just use the number ======================================= In yoga cycle, we changed the release naming convention to add the number also in <year>.<release count in that year> format[11]. But the release name is less fun nowadays and more of a conflict or objection from the community. Starting from Ussuri, I have seen objections on many release names and the most recent is 'Zed'. Being one of the release name process coordinators for the last couple of releases, I feel that I am doing a good job and have to accept the blame/objection. Also, the process of selecting the release name has its own cost in terms of TC, community time as well as the cost of the legal check by the foundation. Having only the release number will help to understand how old that release is and also in understanding the tick-tock releases. The Drawback of removing the release name is that it requires changes in release tooling and will be less helpful in marketing. The former one is a one-time change and later once can be adjusted with having some tag lines. For example, "'OpenStack 2022.2': The integration of your effort" Considering all the facts, TC agreed to start the release name drop process in Gerrit and the community can add their feedback on that. Gate Health and Checks =================== This is something TC started monitoring and helping on gate health and it is going in a good direction. TC will continue doing it in every weekly meeting. We encourage every project to keep monitoring their rechecks and TC also will keep eye on those when we notice a large number of rechecks without comment. * Infra resource checks: The next thing we checked was if there are enough infra resources we have to have the smooth development of OpenStack in the Zed cycle. Clark and fungi gave updates on those and we are all in a good position at least not in a panic situation. But to monitor the required infra resources correctly, TC will work on listing the required and good-to-have services we need for OpenStack development. That way we will be able to judge the infra resource situation in a better way. * ELK service: The new dashboard for ELK service is ready to use, you can find the dashboard and credential information in this review[12]. We encourage community members to start using it and provide feedback. Accordingly, Clark will plan to stop the older ELK servers in a month or so. Thanks dpawlik for working on it. * nested-virt capable instances for testing We discussed it and they are available and can be requested from the job definition but there is always a risk of a slow job and timing failure and it is preferred to use those in the project gate only and not in the integrated/cross-project gate, Testing strategy: ============= * Direction on the lower constraints As lower constraints are not well tested and always causing issues in upstream CI, we are discussing it again to find some permanent solution. We discussed both cases 1. keeping lower bound in requirements.txt files and 2. lower-constraints.txt file and its testing. We agree to keep the 1st one but drop the 2nd one completely. Below is the agreement and action items: ** AGREE/ACTION ITEM: *** Write up how downstream distro can test their constraint file with OpenStack upstream. *** Keep the lower bound in requirements.txt but add a line saying that they are the best effort but not tested. If they are wrong then file a bug or fix it. *** Drop the lower-constraints.txt file, l-c tox env and its testing on master as well as on all stable branches. TC will add the resolution and communicate that on ML too. * Release specific job template for testing runtime: As brought up in ML[13], release-specific template does not actually work for independent releases model repo where the OpenStack bot does not update the template. Stephen added a new generic template to use in such repo. But if we see these release-specific templates are adding an extra step to update in every repo on a new release. Having a generic template and handling the python version jobs with branch variant[14] will work fine. But the generic template will move all the projects to new testing runtime at the same time from a central place. We agree to do it but with proper communication to all the projects and give some window to test/fix them before we switch to new testing versions. * Performance testing/monitoring Performance testing is good to do but as per our current CI resources, it is very difficult. We discussed few ideas where we can keep eyes on the performance aspect: ** At the start, we can monitor the memory footprint via performance stats in CI jobs ** Before and after DB query counting (how does this patch affect the number of DB queries) ** For API performance, we can monitor all the API requests via a single behind tls proxy gateway and collect the stats. * When to remove the deprecated devstack-gate? We agree to remove it once stable/wallaby is EM and will update the same in README file also with the expected date of stable/wallaby to be EM (2022-10-14). Improvement in project governance (continue from Yoga PTG....) ================================================ This is regarding how we can better keep eyes on the less or no active projects and detect them earlier in the cycle to same the release, infra resources. In the Yoga release, we faced the issue of a few projects being broken during the last date of the final release. We discussed it in the last PTG and agreed to start the 'tech-preview' idea. To define what can be entry and exit criteria for any project to be in 'tech-preview', we need input from the release team on stopping the auto release of projects or so. The release team was busy in their PTG at the same time we had this discussion so I will continue the discussion in TC weekly meetings. Current state of translations in OpenStack ================================ Ianychoi and Seongsoocho attended this session and gave us a detailed picture of the translation work, what is the current state and what all work is required to do including migration from Zanata to weblate. The main issue here is we need more people helping in this work. We also asked who uses those translations and they should come up to help. As the next step, we agreed to add a new question to the user survey ("Do you use i18n translations in your deployments? If yes please tell us which languages you use") and get some stats about translation usage. Meanwhile, Ian and seongsoocho team will continue maintaining. Thanks to Ian and Seongsoocho for their best effort to maintain it. Cross community sessions with the k8s steering committee team: ================================================= k8s steering committee members (Paris, Dims, Bob, and Tim) joined the OpenStack Technical Committee in PTG. We discussed the various topics about Legal support/coverage for contributors, especially in the security disclosure process and export control. We asked if k8s also have the same level of language translation support in code/doc as that we have in OpenStack and they have only for the documentation which is also if anyone proposes to do. Then we discussed the long-term sustainability efforts, especially for the experience contributors who can take higher responsibility as well as train the new contributors. This is an issue in both communities and none of us have any solution to this. In the last, we discussed how k8s recognize the contributor's good work. In k8s along with appreciating on slack, ML, they issue the coupon and badges from the CNCF foundation. TC Zed Activities checks and planning ============================= This is the last hour of the PTG and we started with the yoga cycle retrospective. * Yoga Retrospective We are doing good in gate health monitoring as well doing more technical work. On the improvement side, we need to be faster at making the decision on things that are in discussion for long period instead of keeping them open. * Pop Up Team Check In After checking with the status and need of both active popup teams, we decided to continue with both in Zed cycle. * TC weekly Meeting time check We will keep the current time until daylight saving time changes again. Also, we will continue the video call once a month. * TC liaison continue or drop? As we have improved the interaction with the project with the weekly meetings as well as in PTG sessions (TC+Leaders interaction session), we agreed to drop the TC liaisons completely. * I will prepare the Zed cycle TC Tracker (activities we will do in Zed cycle) * Next week's TC meeting is cancelled and we will resume meeting from 21st April onwards. That is all from PTG, thanks for reading it and stay safe! [1] https://docs.openstack.org/project-team-guide/technical-guides/unified-limi… [2] https://review.opendev.org/c/openstack/governance/+/816387 [3] https://review.opendev.org/c/openstack/governance/+/835102 [4] https://docs.openstack.org/project-team-guide/spread-the-word.html [5] https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-te… [6] https://review.opendev.org/c/openstack/governance/+/836888 [7] http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h… [8] https://etherpad.opendev.org/p/qa-zed-ptg-fips [9] https://etherpad.opendev.org/p/zed-ptg-fips-goal [10] https://docs.openstack.org/project-team-guide/deprecation.html [11] https://review.opendev.org/c/openstack/governance/+/829563 [12] https://review.opendev.org/c/openstack/governance-sigs/+/835838 [13] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027676.ht… [14] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/833892

1 0

[release] Release countdown for week R-25, Apr 11 - 15
by Előd Illés 09 Apr '22

09 Apr '22

Hi, Welcome back to the release countdown emails! These will be sent at major points in the Zeddevelopment cycle, which should conclude with a final release on October 5th, 2022. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the Zeddevelopment cycle, assessing Zedcommunity goals and approving Zedspecs. General Information ------------------- Zed is a 27 weeks long development cycle. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/ <https://releases.openstack.org/zed/schedule.html>zed <https://releases.openstack.org/zed/schedule.html>/schedule.html <https://releases.openstack.org/zed/schedule.html> By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the Yogarelease are continued in Zedwith a similar release model. Upcoming Deadlines & Dates -------------------------- Zed-1 milestone: May 19th, 2022 OpenInfra Summit: June 7-9, 2022 (Berlin) Előd Illés irc: elodilles

1 0

I sent e-mail to this mailing list but got reply from support@paribet.ru! Why!?!???
by 박경원 08 Apr '22

08 Apr '22

2 1

[Openstack][nova] server groups
by Ignazio Cassano 08 Apr '22

08 Apr '22

Hello, we noted that instances can be inserted in server groups only at instance creation step but we need to insert in a server group some old instances. We tried to modify database nova_api server group tables but we noted that we must modify spec in request_specs table . For us is not clear how to modify the spec value. We tried to investigate looking at instances inserted in a server group at creation step and we got issues in instance live migration. Please, anyone could provide any utility to do it or any template ? Thanks Ignazio

2 8

[Question] Do I must separate management network and overlay network?
by 박경원 08 Apr '22

08 Apr '22

3 2

[stable][horizon] add vishalmanchanda and tmazur to horizon-stable-core
by Akihiro Motoki 08 Apr '22

08 Apr '22

Hi, To horizon team, During the horizon PTG, we visit our stable branch reviewers. We (all existing core reviewers) agree to add vishalmanchanda and tmazur to the horizon stable core team. We believe they are familiar with the stable policy. Welcome! We also agreed to drop David Lyle from the horizon stable core as he is no longer active in horizon for long. Thanks David for your contributions so far! To stable-maint-core team, Could you apply the following changes to horizon-stable-main group in the Gerrit? Additions: manchandavishal143(a)gmail.com, t.v.ovtchinnikova(a)gmail.com Removal: dklyle0(a)gmail.com Thanks, Akihiro Motoki (irc: amotoki)

4 3

Re: [neutron] exposing ip address of external Network from
by A Monster 08 Apr '22

08 Apr '22

When I attach an interface to an external network, openstack gives an ip address of 192.168.10.13 to the instance, Inside the instance, I can see a new network interface, but it doesn't get an ip address automatically, and when I configure it to have 192.169.10.13 manually, I still cannot ping the other ips on this network, and I cannot ping 192.168.10.13 from the outside I think it's not feasible to attach instances to external networks.

2 1

[tacker] Cancel PTG Day4 on 8th Apr.
by TAKAHASHI TOSHIAKI(高橋　敏明) 08 Apr '22

08 Apr '22

Hi, We will cancel Tacker PTG Day4 on 8th April because all topics have been discussed and completed. Regards, Toshiaki

2 1

[glance] Outreachy 2022
by Anurag Singh Rajawat 08 Apr '22

08 Apr '22

Dear glance team, I'd setup glance, glance-store and glance-client on my local setup, but some tests for glance were failing, also is there any good first issues so that I can understand the project more clearly? I also asked about it in IRC but doesn't got response. Thanks Sincerely Anurag

2 1

[glance] Zed PTG schedule
by Abhishek Kekane 08 Apr '22

08 Apr '22

Hello All, Greetings!!! Zed PTG is going to start next week and if you haven't already registered, please do so as soon as possible [1]. I have created an etherpad [2] and also added day wise topics along with timings we are going to discuss. Kindly let me know if you have any concerns with allotted time slots. We also have one slot open on Wednesday and Friday is kept reserved for any unplanned discussions. So please feel free to add your topics if you still haven't added yet. As a reminder, these are the time slots for our discussion. Tuesday 5 April 2022 1400 UTC to 1700 UTC Wednesday 6 April 2022 1400 UTC to 1700 UTC Thursday 7 April 2022 1400 UTC to 1700 UTC Friday 8 April 2022 1400 UTC to 1700 UTC NOTE: At the moment we don't have any sessions scheduled on Friday, if there are any last moment request(s)/topic(s) we will discuss them on Friday else we will conclude our PTG on Thursday 7th April. We will be using bluejeans for our discussion, kindly try to use it once before the actual discussion. The meeting URL is mentioned in etherpad [2] and will be the same throughout the PTG. [1] https://openinfra-ptg.eventbrite.com/ [2] https://etherpad.opendev.org/p/zed-glance-ptg Thank you, Abhishek

2 2

OPS Meetup Call for Topics
by Amy Marrich 08 Apr '22

08 Apr '22

During our PTG session, we started the process of gathering potential topics for the OPS Meetup to be held in Berlin on June 10. If you are interested in attending the event ,please add any topics you wish to discuss to the etherpad[0]. Deadline for adding new topics is April 30, so we can open for session voting in early May. Registration will be available shortly. Thanks, Amy 0 - https://etherpad.opendev.org/p/april2022-ptg-openstack-ops

1 0