- openstack-discuss - lists.openstack.org

how-to enable s3api
by admin 05 Sep '20

05 Sep '20

hi where is there a normal manual on how to enable s3api support in swift? trying to go through https://docs.openstack.org/swift/latest/middleware.html and as a result keystone gives an error in authorization (trying to connect via (s3curl) Authorization failed. The request you have made requires authentication. from x.x.x.x Unauthorized: The request you have made requires authentication.

2 1

[mq][mariadb] what kind of reasons are considered when choosing the mq/mariadb version
by Sheldon Hu(胡玉鹏) 04 Sep '20

04 Sep '20

Hi all I use kolla to build the openstack version, use kolla-ansible to deploy rabbitmq is v3.7.10, mariadb is v10.1.x when building rocky version rabbitmq is v3.8.5, mariadb is v10.3.x when building ussuri version what kind of reasons are considered as rabbitmq version is changed from v3.7.10 to v3.8.5 ? what kind of reasons are considered as mariadb version is changed from v10.1.x to v10.3.x ? many thanks. Sheldon Hu | 胡玉鹏 CBRD | 云计算与大数据研发部 T: 18663761785 E: <mailto:huyp@inspur.com> huyp(a)inspur.com 浪潮电子信息产业股份有限公司 Inspur Electronic Information Industry Co.,Ltd. 山东省济南市历城区东八区企业公馆A7-1 Building A7-1, Dongbaqu Office Block, Licheng District, Jinan,Shandong Province, PRC 浪潮云海

2 1

[scientific]
by Foundjem Armstrong 04 Sep '20

04 Sep '20

Hello Blair, I am wondering of this email is still functional. Regards, Armstrong

2 1

[TC] Looking for a volunteer to help with community goal selection
by Nate Johnston 04 Sep '20

04 Sep '20

TC members, Since I have not been able to get in contact with Graham, I would like to see if there is anyone who would like to work with me for the selection of community goals for the W series. The commitment would be to identify a few good candidates that have well formulated goal proposals and champions, which usually involves a meeting followed by some reaching out and logistical work. I am hoping to increase the pool of those familiar with the goal selection process, so this is a great place to jump in if you have not done this before. Also, if Graham does have time then we can make sure to include him as well. Two is good but three is better! Please let me know if you are interested. Thanks! Nate

1 0

Mentoring Boston University Students
by Kendall Nelson 04 Sep '20

04 Sep '20

Hello! As you may or may not know, the last two years various community members have mentored students from North Dakota State University for a semester to work on projects in OpenStack. Recently, I learned of a similar program at Boston University and they are still looking for mentors interested for the upcoming semester. Essentially you would have 5 to 7 students for 13 weeks to mentor and work on some feature or effort in your project. The time to apply is running out however as the deadline is Sept 3rd. If you are interested, please let me know ASAP! I am happy to help get the students up to speed with the community and getting their workspaces set up, but the actual work they would do is more up to you :) -Kendall (diablo_rojo)

2 3

[tripleo] centos-binary -> openstack-
by Wesley Hayutin 04 Sep '20

04 Sep '20

Greetings, The container names in master have changed from centos-binary* to openstack*. https://opendev.org/openstack/tripleo-common/src/branch/master/container-im… https://opendev.org/openstack/tripleo-common/commit/90f6de7a7fab15e9161c1f0… If your patches are failing to pull https://registry-1.docker.io/v2/tripleomaster/centos-binary* it's not going to be fixed in a recheck. Check that your patches are rebased and your dependent patches are rebased. Thanks 0/

4 3

[oslo][ffe] request for oslo
by Herve Beraud 04 Sep '20

04 Sep '20

Hey Oslofolk, I request an FFE for these oslo.messaging changes [1]. The goal of these changes is to run rabbitmq heartbeat in python thread by default. Also these changes deprecating this option to prepare future removal and force to always run heartbeat in a python thread whatever the context. Land these changes during the victoria cycle can help us to prime the option removal during the next cycle. Thanks for your time, [1] https://review.opendev.org/#/c/747395/ -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

3 3

[all][TC] OpenStack Client (OSC) vs python-*clients
by Belmiro Moreira 04 Sep '20

04 Sep '20

Hi, during the last PTG the TC discussed the problem of supporting different clients (OpenStack Client - OSC vs python-*clients) [1]. Currently, we don't have feature parity between the OSC and the python-*clients. Different OpenStack projects invest in different clients. This can be a huge problem for users/ops. Depending on the projects deployed in their infrastructures, they need to use different clients for different tasks. It's confusing because of the partial implementation in the OSC. There was also the proposal to enforce new functionality only in the SDK (and optionally the OSC) and not the project’s specific clients to stop increasing the disparity between the two. We would like to understand first the problems and missing pieces that projects are facing to move into OSC and help to overcome them. Let us know. Belmiro, on behalf of the TC [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015418.html

13 20

[nova] Session Recording for Feedback
by Apsey, Christopher 04 Sep '20

04 Sep '20

Nova supports using the websockify -record functionality to capture frames sent over arbitrary console sessions. Currently, the record = path/to/file option in nova.conf saves these sessions on nova-(spice|vnc)proxy endpoints at the specified path in the format of /path/to/file.(session number). These session numbers appear to be incrementally created starting from 1 and don't really have an association with their respective instance from what I can tell. It would be useful to have these files saved using the instance UUID instead. Ignoring that minor problem, playing back these files is a challenge. I've tried using noVNCs vnc_playback.html function from both the master branch and stable/v0.6 with no luck - it looks like the libraries and the utilities haven't been maintained at the same cadence and there are missing functions that the player is expecting. My goal here is to be able to capture the activities of students as they progress through various exercises and then have instructors be able to go over their work with them after the fact if they have questions. Has anyone been able to do this (or something like this) successfully, or are we just better off trying to do this in-band per instance? Chris Apsey GEORGIA CYBER CENTER

2 1

Re: Kuryr openstack
by mahdi abbasi 03 Sep '20

03 Sep '20

Thanks a lot This issue has been resolved On Thu, 3 Sep 2020, 23:28 Radosław Piliszek, <radoslaw.piliszek(a)gmail.com> wrote: > Hi Mahdi, > > have you created any networks (as in Neutron resource) in that project? > You need to have at least one network for Zun to use. > > -yoctozepto > > On Thu, Sep 3, 2020 at 6:50 PM mahdi abbasi <mahdi.abbasi.2013(a)gmail.com> > wrote: > > > > Hi development team, > > > > Do i need a special configuration to use kuryr when using openvswitch? > > I get the following error when starting Docker container in zun: > > > > Unable to create the network.No tenant network is availble for > allocation. > > > > Please help me. > > > > Best regards > > Mahdi >

1 0

Kuryr openstack
by mahdi abbasi 03 Sep '20

03 Sep '20

Hi development team, Do i need a special configuration to use kuryr when using openvswitch? I get the following error when starting Docker container in zun: Unable to create the network.No tenant network is availble for allocation. Please help me. Best regards Mahdi

2 2

[InteropWG] Brainstorming and looking for Ideas from cross project teams
by prakash RAMCHANDRAN 03 Sep '20

03 Sep '20

Hi all, Interop WG i seeking cross platform participation to enable Baremetal and Kubernetes ready Modules in OpenStack & Open Infrastructure. Please add what you would like to get help from Interop Group and how in turn you can help Interop WG goals of enabling Multi-Cloud & Hybrid Cloud Logo Programs to be initiated in CY Q4 2019. We like to discuss and get any new ideas whetted and supported by community. Please add and join us in Forum, PTG and Summit. https://etherpad.opendev.org/p/2020-Wallaby-interop-brainstorming Look forward to your Projects valued participation in Interop Branding efforts.Our next Interop Call is shceduled for Sept 11th 10 AM PDT, please add before that all you can bring to table. For Interop WG Prakash RamchandranIndi Member BoD

1 0

[oslo] Feature Freeze
by Ben Nemec 03 Sep '20

03 Sep '20

This is just a heads up that Oslo is now in feature freeze. Any changes that would require a feature release of an Oslo library will need to request an FFE if they want to make the Victoria release. Please send an email tagged "[oslo][ffe]" to the list to make such a request. I expect we'll branch the Oslo repos in the near future, so some feature development can continue then. However, we generally prefer not to merge invasive changes between feature freeze and release just in case a last-minute bugfix backport is needed. Disentangling a bug fix from a major feature could be a chore. Thanks. -Ben

1 0

[neutron] Drivers meeting - 04.09.2020
by Slawek Kaplonski 03 Sep '20

03 Sep '20

Hi, As there is no agenda for tomorrow's meeting, I will cancel it. Lets focus on reviewing patches related to the RFE scheduled for Victoria-3: https://launchpad.net/neutron/+milestone/victoria-3 See You all online and have a great weekend o/ -- Slawek Kaplonski Principal software engineer Red Hat

1 0

Issue with libvirt unable to kill processes
by Grant Morley 03 Sep '20

03 Sep '20

Hi All, I was wondering if anyone has come across an issue with libvirt seemingly having an issue with instances all of a sudden "locking up" with the following error: Failed to terminate process 2263874 with SIGKILL: Device or resource busy In the nova logs I am seeing: 2020-09-03 09:13:43.208 2659995 INFO nova.compute.manager [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Rebooting instance 2020-09-03 09:15:52.429 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Failed to soft reboot instance. Trying hard reboot. 2020-09-03 09:16:32.450 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy. Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or resource busy; attempt 1 of 3: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:12.484 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy. Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or resource busy; attempt 2 of 3: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:52.516 2659995 WARNING nova.virt.libvirt.driver [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Error from libvirt during destroy. Code=38 Error=Failed to terminate process 2263874 with SIGKILL: Device or resource busy; attempt 3 of 3: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:52.526 2659995 ERROR nova.compute.manager [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Cannot reboot instance: Failed to terminate process 2263874 with SIGKILL: Device or resource busy: libvirtError: Failed to terminate process 2263874 with SIGKILL: Device or resource busy 2020-09-03 09:17:53.026 2659995 INFO nova.compute.manager [req-7ffb9b7c-799b-40dc-be04-598bfda2e2fc 6ddb647baf9343b09d7f8f7a32b0b43d 8cce5e1532e6435a90b168077664bbdf - default default] [instance: f3a8c916-28f5-432d-9b8e-c3056d2dee5a] Successfully reverted task state from reboot_started on failure for instance. It seems to be caused when a reboot happens to an instance. If you reset the state and try again, the same error occurs. You also seemingly cannot kill off any libvirt process that is attached to that instance. To me it looks like it could be a kernel issue with libvirt but I could be wrong? Does anyone know of a workaround for this other than maybe restarting a compute host? Many thanks,

2 1

Kuryr openstack
by mahdi abbasi 03 Sep '20

03 Sep '20

Hi development team, Do i need a special configuration to use kuryr when using openvswitch? I get the following error when starting Docker container in zun. Please help me. Best regards Mahdi

1 0

[openstack-ansible] OpenStack Ansible deployment fails due to lxc containers not having network connection
by Oliver Wenz 03 Sep '20

03 Sep '20

I'm trying to deploy OpenStack Ansible. When running the first playbook ```openstack-ansible setup-hosts.yml```, there are errors for all containers during the task ```[openstack_hosts : Remove the blacklisted packages]``` (see below) and the playbook fails. ``` fatal: [infra1_repo_container-1f1565cd]: FAILED! => {"changed": false, "cmd": "apt-get update", "msg": "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release' no longer has a Release file.", "rc": 100, "stderr": "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release' no longer has a Release file. E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release' no longer has a Release file. ", "stderr_lines": ["E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic Release' no longer has a Release file.", "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release' no longer has a Release file.", "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release' no longer has a Release file.", "E: The repository 'http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release' no longer has a Release file."], "stdout": "Ign:1 http://ubuntu.mirror.lrz.de/ubuntu bionic InRelease Ign:2 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates InRelease Ign:3 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports InRelease Ign:4 http://ubuntu.mirror.lrz.de/ubuntu bionic-security InRelease Err:5 http://ubuntu.mirror.lrz.de/ubuntu bionic Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Err:6 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Err:7 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Err:8 http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable) Reading package lists... ", "stdout_lines": ["Ign:1 http://ubuntu.mirror.lrz.de/ubuntu bionic InRelease", "Ign:2 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates InRelease", "Ign:3 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports InRelease", "Ign:4 http://ubuntu.mirror.lrz.de/ubuntu bionic-security InRelease", "Err:5 http://ubuntu.mirror.lrz.de/ubuntu bionic Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Err:6 http://ubuntu.mirror.lrz.de/ubuntu bionic-updates Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Err:7 http://ubuntu.mirror.lrz.de/ubuntu bionic-backports Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Err:8 http://ubuntu.mirror.lrz.de/ubuntu bionic-security Release", " Cannot initiate the connection to 192.168.100.6:8000 (192.168.100.6). - connect (101: Network is unreachable)", "Reading package lists..."]} ``` When I attach to any container and run ```ping 192.168.100.6``` (local DNS), I get the same error (```connect: Network is unreachable```). However, when I specify an interface by running ```ping -I eth1 192.168.100.6``` there is a successful connection. Running ```ip r``` on the infra_cinder container yields: ``` 10.0.3.0/24 dev eth2 proto kernel scope link src 10.0.3.5 192.168.110.0/24 dev eth1 proto kernel scope link src 192.168.110.232 ``` so there seems to be no default route which is why the connection fails (similar for the other infra containers). Shouldn't OSA automatically configure this? I didn't find anything regarding a default route on containers in the Docs. Here's my openstack_user_config.yml: ``` cidr_networks: container: 192.168.110.0/24 tunnel: 192.168.32.0/24 storage: 10.0.3.0/24 used_ips: - "192.168.110.1,192.168.110.2" - "192.168.110.111" - "192.168.110.115" - "192.168.110.117,192.168.110.118" - "192.168.110.131,192.168.110.140" - "192.168.110.201,192.168.110.207" - "192.168.32.1,192.168.32.2" - "192.168.32.201,192.168.32.207" - "10.0.3.1" - "10.0.3.11,10.0.3.14" - "10.0.3.21,10.0.3.24" - "10.0.3.31,10.0.3.42" - "10.0.3.201,10.0.3.207" global_overrides: # The internal and external VIP should be different IPs, however they # do not need to be on separate networks. external_lb_vip_address: 192.168.100.168 internal_lb_vip_address: 192.168.110.201 management_bridge: "br-mgmt" provider_networks: - network: container_bridge: "br-mgmt" container_type: "veth" container_interface: "eth1" ip_from_q: "container" type: "raw" group_binds: - all_containers - hosts is_container_address: true - network: container_bridge: "br-vxlan" container_type: "veth" container_interface: "eth10" ip_from_q: "tunnel" type: "vxlan" range: "1:1000" net_name: "vxlan" group_binds: - neutron_linuxbridge_agent - network: container_bridge: "br-ext1" container_type: "veth" container_interface: "eth12" host_bind_override: "eth12" type: "flat" net_name: "ext_net" group_binds: - neutron_linuxbridge_agent - network: container_bridge: "br-storage" container_type: "veth" container_interface: "eth2" ip_from_q: "storage" type: "raw" group_binds: - glance_api - cinder_api - cinder_volume - nova_compute - swift-proxy ### ### Infrastructure ### # galera, memcache, rabbitmq, utility shared-infra_hosts: infra1: ip: 192.168.110.201 # repository (apt cache, python packages, etc) repo-infra_hosts: infra1: ip: 192.168.110.201 # load balancer haproxy_hosts: infra1: ip: 192.168.110.201 ### ### OpenStack ### os-infra_hosts: infra1: ip: 192.168.110.201 identity_hosts: infra1: ip: 192.168.110.201 network_hosts: infra1: ip: 192.168.110.201 compute_hosts: compute1: ip: 192.168.110.204 compute2: ip: 192.168.110.205 compute3: ip: 192.168.110.206 compute4: ip: 192.168.110.207 storage-infra_hosts: infra1: ip: 192.168.110.201 storage_hosts: lvm-storage1: ip: 192.168.110.202 container_vars: cinder_backends: lvm: volume_backend_name: LVM_iSCSI volume_driver: cinder.volume.drivers.lvm.LVMVolumeDriver volume_group: cinder_volumes iscsi_ip_address: "{{ cinder_storage_address }}" limit_container_types: cinder_volume ``` I also asked this question on the server fault stackexchange: https://serverfault.com/questions/1032573/openstack-ansible-deployment-fail… Kind regards, Oliver

2 1

[API SIG] Ending office hours for the API-SIG
by Michael McCune 03 Sep '20

03 Sep '20

hello all, the API-SIG has held meetings and office hours for several years now. since our migration from a weekly meeting to an office hour we have monitored a continual decrease in attendance for these hours, aside from the core membership. after discussion over the course of the last month or two, we have agreed to end the API-SIG office hours. we will hold office hours this week as our final meeting. if you need to contact the SIG in the future, we are still available in the #openstack-sdks channel on freenode as well as here on the mailing list. in the future we would like to see continued migration of the API-SIG closer to the SDKs group. these two groups will essentially become a singular place to discuss all things related to SDKs and APIs in OpenStack. thank you all for the years of participation and discussion that helped us as a community to define a set of best practices and guidelines for API work in OpenStack. peace o/

1 0

senlin auto scaling question
by Satish Patel 03 Sep '20

03 Sep '20

Folks, I have created very simple cluster using following command openstack cluster create --profile myserver --desired-capacity 2 --min-size 2 --max-size 3 --strict my-asg It spun up 2 vm immediately now because the desired capacity is 2 so I am assuming if any node dies in the cluster it should spin up node to make count 2 right? so i killed one of node with "nove delete <instance-foo-1>" but senlin didn't create node automatically to make desired capacity 2 (In AWS when you kill node in ASG it will create new node so is this senlin different then AWS?)

3 5

[kolla] questions when using external mysql
by sosogh 03 Sep '20

03 Sep '20

Hi list: I want to use kolla-ansible to deploy openstack , but using external mysql. I am following these docs: https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html https://docs.openstack.org/kolla-ansible/latest/reference/databases/externa…. I have some questions: ################ ## Question 1 ## ################ According to the offical doc , if setting it in inventory file(multinode), kolla-ansible -i ./multinode deploy will throw out error: I guest when kolla-ansible running the playbook against myexternalmariadbloadbalancer.com , the """register: find_custom_fluentd_inputs""" in """TASK [common : Find custom fluentd input config files]""" maybe null . ################ ## Question 2 ## ################ According to the offical doc , If the MariaDB username is not root, set database_username in /etc/kolla/globals.yml file: But in kolla-ansible/ansible/roles/xxxxxx/tasks/bootstrap.yml , they use ''' login_user: "{{ database_user }}" ''' ， for example : So at last , I took the following steps: 1. """not""" setting [mariadb] in inventory file(multinode) 2. set "database_user: openstack" for "privillegeduser" PS: My idea is that if using an external ready-to-use mysql (cluster), it is enough to tell kolla-ansible only the address/user/password of the external DB. i.e. setting them in the file /etc/kolla/globals.yml and passwords.yml , no need to add it into inventory file(multinode) Finally , it is successful to deploy openstack via kolla-ansible . So far I have not found any problems. Are the steps what I took good ( enough ) ? Thank you !

2 4

[tacker] Wallaby vPTG
by Yasufumi Ogawa 02 Sep '20

02 Sep '20

Hi team, I booked our slots, 27-29 Oct 6am-8am UTC, for the next vPTG[1] as we agreed in previous irc meeting. I also prepared an etherpad [2], so please add your name and suggestions. [1] https://ethercalc.openstack.org/7xp2pcbh1ncb [2] https://etherpad.opendev.org/p/Tacker-PTG-Wallaby Thanks, Yasufumi

2 2

[ovn][neutron][ussuri] How to configure for overlay in DPDK with OVN ?
by 한승진 02 Sep '20

02 Sep '20

Hi, I’m trying to implement dpdk-ovs with ovn for our openstack environment. I create a bond interface(tunbound) with ovs and tun0 and tun1 are slave interfaces of the bond interface named tunbond. Bridge br-int fail_mode: secure datapath_type: netdev Port tunbond Interface tun0 type: dpdk options: {dpdk-devargs="0000:1a:00.1", n_rxq="2"} Interface tun1 type: dpdk options: {dpdk-devargs="0000:d8:00.1", n_rxq="2"} Port br-int Interface br-int type: internal Now, I need to configure remote-ovn options to the ovs. *I am confused how can I define the IP address of the bonded interface with ovs?* Theses are commands for connect remote ovn services. ovs-vsctl set open . external-ids:ovn-remote=tcp:{controller ip}:6642 ovs-vsctl set open . external-ids:ovn-encap-type=geneve *1* This below command make it me confused. ovs-vsctl set open . external-ids:ovn-encap-ip={local ip} How should I resolve this issue? Regards, John Haan

2 2

[tripleo, ironic] Error: Could not retrieve ... pxelinux.0
by Pete Zaitcev 02 Sep '20

02 Sep '20

Hello: I wanted to give the TripleO a try, so started follow our installation guide for Ussuri, and eventually made it to "openstack undercloud install". It fails with something like this: Aug 28 10:10:53 undercloud puppet-user[48657]: Error: /Stage[main]/Ironic::Pxe/File[/var/lib/ironic/tftpboot/ipxe.efi]: Could not evaluate: Could not retrieve information from environment production source(s) file:/usr/share/ipxe/ipxe-x86_64.efi Aug 27 20:05:42 undercloud puppet-user[37048]: Error: /Stage[main]/Ironic::Pxe/Ironic::Pxe::Tftpboot_file[pxelinux.0]/File[/var/lib/ironic/tftpboot/pxelinux.0]: Could not evaluate: Could not retrieve information from environment production source(s) file:/tftpboot/pxelinux.0 Does anyone have an idea what it wants? I added a couple of packages on the host system that provided the files mentioned in the message, but it made no difference. Ussuri is conteinerized anyway. Since I'm very new to this, I have no clue where to look at all. The nearest task is a wrapper of some kind, so the install-undercloud.log looks like this: 2020-08-28 14:11:31.397 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] TASK [Run container-puppet tasks (generate config) during step 1 with paunch] *** 2020-08-28 14:11:31.397 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] Friday 28 August 2020 14:11:31 -0400 (0:00:00.302) 0:06:28.734 ********* 2020-08-28 14:11:32.223 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] changed: [undercloud] 2020-08-28 14:11:32.325 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] 2020-08-28 14:11:32.326 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] TASK [Wait for container-puppet tasks (generate config) to finish] ************* 2020-08-28 14:11:32.326 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] Friday 28 August 2020 14:11:32 -0400 (0:00:00.928) 0:06:29.663 ********* 2020-08-28 14:11:32.948 60599 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ ] WAITING FOR COMPLETION: Wait for container-puppet tasks (generate config) to finish (1200 retries left). . . . If anyone could tell roughly what is supposed to be going on here, it would be great. I may be able figure out the rest. Greetings, -- Pete

2 3

Re: [neutron][ovn] OVN Performance
by Apsey, Christopher 02 Sep '20

02 Sep '20

All, Just wanted to loop back here and give an update. For reference, [1] (blue means successful action, red means failed action) is the result we got when booting 5000 instances in rally [2] before the Red Hat OVN devs poked around inside our environment, and [3] is the result after. The differences are obviously pretty significant. I think the biggest change was setting metadata_workers = 2 in neutron_ovn_metadata_agent.ini on the compute nodes per https://bugs.launchpad.net/neutron/+bug/1893656. We have 64C/128T on all compute nodes, so the default neutron calculation of scaling metadata workers based on available cores created 900+ connections to the southbound db at idle; after the control plane got loaded up it just quit around 2500 instances (my guess is it hit the open file limit, although I don’t think increasing it would have made it better for much longer since the number of connections were increasing exponentially). Capping the number of metadata workers decreased open southbound connections by 90%. Even more telling was that rally was able to successfully clean up after itself after we made that change, whereas previously it wasn’t even able to successfully tear down any of the instances that were made, indicating that the control plane was completely toast. Note that the choppiness towards the end of [3] had nothing to do with OVN – our compute nodes had a loadavg approaching 1000 at that point, so they were just starved for cpu cycles. This would have scaled even better with additional compute nodes. The other piece was RAFT. Currently, RDO is shipping with ovs 2.12, but 2.13 has a bunch of RAFT fixes in it that improve stability and knock out some bugs. We were having issues with chassis registration on 2.12, but after using the 2.13 package from cbs, all those issues went away. Big thanks to the great people at Red Hat on the cc line for volunteering their valuable time to take a look. I’m now significantly more comfortable with defaulting to OVN as the backend of choice as the performance delta is now gone. That said, should the community consider dropping linuxbridge as the backend in the official upstream docs and jump straight to OVN rather than ml2/OVS? I think that would increase the test base and help shine light on other issues as time goes on. My org can devote some time to doing this work if the community agrees that it’s the right action to take. Hope that’s helpful! [1] https://ibb.co/GTjZP2y [2] https://pastebin.com/5pEDZ7dY [3] https://ibb.co/pfB9KTV Chris Apsey GEORGIA CYBER CENTER From: Apsey, Christopher Sent: Thursday, August 27, 2020 11:33 AM To: Assaf Muller <amuller(a)redhat.com> Cc: openstack-discuss(a)lists.openstack.org; Lucas Alvares Gomes Martins <lmartins(a)redhat.com>; Jakub Libosvar <jlibosva(a)redhat.com>; Daniel Alvarez Sanchez <dalvarez(a)redhat.com> Subject: RE: [EXTERNAL] Re: [neutron][ovn] OVN Performance Assaf, We can absolutely support engineering poking around in our environment (and possibly an even larger one at my previous employer that was experiencing similar issues during testing). We can take this offline so we don’t spam the mailing list. Just let me know how to proceed, Thanks! Chris Apsey GEORGIA CYBER CENTER From: Assaf Muller <amuller(a)redhat.com<mailto:amuller@redhat.com>> Sent: Thursday, August 27, 2020 11:18 AM To: Apsey, Christopher <CAPSEY(a)augusta.edu<mailto:CAPSEY@augusta.edu>> Cc: openstack-discuss(a)lists.openstack.org<mailto:openstack-discuss@lists.openstack.org>; Lucas Alvares Gomes Martins <lmartins(a)redhat.com<mailto:lmartins@redhat.com>>; Jakub Libosvar <jlibosva(a)redhat.com<mailto:jlibosva@redhat.com>>; Daniel Alvarez Sanchez <dalvarez(a)redhat.com<mailto:dalvarez@redhat.com>> Subject: [EXTERNAL] Re: [neutron][ovn] OVN Performance CAUTION: EXTERNAL SENDER This email originated from an external source. Please exercise caution before opening attachments, clicking links, replying, or providing information to the sender. If you believe it to be fraudulent, contact the AU Cybersecurity Hotline at 72-CYBER (2-9237 / 706-722-9237) or 72CYBER(a)augusta.edu<mailto:72CYBER@augusta.edu> The most efficient way about this is to give one or more of the Engineers working on OpenStack OVN upstream (I've added a few to this thread) temporary access to an environment that can reproduce issues you're seeing, we could then document the issues and work towards solutions. If that's not possible, if you could provide reproducer scripts, or alternatively sharpen the reproduction method, we'll take a look. What you've described is not something that's 'acceptable', OVN should definitely not scale worse than Neutron with the Linux Bridge agent. It's possible that the particular issues you ran in to is something that we've already seen internally at Red Hat, or with our customers, and we're already working on fixes in future versions of OVN - I can't tell you until you elaborate on the details of the issues you're seeing. In any case, the upstream community is committed to improving OVN scale and fixing scale issues as they pop up. Coincidentally, Red Hat scale engineers just published an article [1] about work they've done to scale RH-OSP 16.1 (== OpenStack Train on CentOS 8, with OVN 2.13 and TripleO) to 700 compute nodes. [1] https://www.redhat.com/en/blog/scaling-red-hat-openstack-platform-161-more-…<https://www.redhat.com/en/blog/scaling-red-hat-openstack-platform-161-more-…> On Thu, Aug 27, 2020 at 10:44 AM Apsey, Christopher <CAPSEY(a)augusta.edu<mailto:CAPSEY@augusta.edu>> wrote: > > All, > > > > I know that OVN is going to become the default neutron backend at some point and displace linuxbridge as the default configuration option in the docs, but we have noticed a pretty significant performance disparity between OVN and linuxbridge on identical hardware over the past year or so in a few different environments[1]. I know that example is unscientific, but similar results have been borne out in many different scenarios from what we have observed. There are three main problems from what we see: > > > > 1. OVN does not handle large concurrent requests as well as linuxbridge. Additionally, linuxbridge concurrent capacity grows (not linearly, but grows nonetheless) by adding additional neutron API endpoints and RPC agents. OVN does not really horizontally scale by adding additional API endpoints, from what we have observed. > > 2. OVN gets significantly slower as load on the system grows. We have observed a soft cap of about 2000-2500 instances in a given deployment before ovn-backed neutron stops responding altogether to nova requests (even for booting a single instance). We have observed linuxbridge get to 5000+ instances before it starts to struggle on the same hardware (and we think that linuxbridge can go further with improved provider network design in that particular case). > > 3. Once the southbound database process hits 100% CPU usage on the leader in the ovn cluster, it’s game over (probably causes 1+2) > > > > It's entirely possible that we just don’t understand OVN well enough to tune it [2][3][4], but then the question becomes how do we get that tuning knowledge into the docs so people don’t scratch their heads when their cool new OVN deployment scales 40% as well as their ancient linuxbridge-based one? > > > > If it is ‘known’ that OVN has some scaling challenges, is there a plan to fix it, and what is the best way to contribute to doing so? > > > > We have observed similar results on Ubuntu 18.04/20.04 and CentOS 7/8 on Stein, Train, and Ussuri. > > > > [1] https://pastebin.com/kyyURTJm<https://pastebin.com/kyyURTJm> > > [2] https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/ovsdb<https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/ovsdb> > > [3] https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/neutron<https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/neutron> > > [4] https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/compute<https://github.com/GeorgiaCyber/kinetic/tree/master/formulas/compute> > > > > Chris Apsey > > GEORGIA CYBER CENTER > >

2 1

[manila] Victoria Collab Review next Tuesday (Sep 1st)
by Douglas 02 Sep '20

02 Sep '20

Hi everybody We will have a new edition of our collaborative review next Tuesday, September 1st, where we'll go through the code and review the proposed feature Share Server Migration[1][2]. This meeting is scheduled for two hours, starting at 5:00PM UTC. Meeting notes and videoconference links will be available here[3]. Feel free to attend if you are interested and available. Hoping to see you there, - dviroel [1] https://opendev.org/openstack/manila-specs/src/branch/master/specs/victoria… [2] https://review.opendev.org/#/q/topic:bp/share-server-migration+(status:open) [3] https://etherpad.opendev.org/p/manila-victoria-collab-review

1 1

Graceful stopping of RabbitMQ AMQP notification listener
by Anil Jangam 02 Sep '20

02 Sep '20

Hi, I have coded OpenStack AMQP listener following the example and it is working fine. https://github.com/gibizer/nova-notification-demo/blob/master/ws_forwarder.… The related code snippets of the NotificationHandler class are shown as follows. # Initialize the AMQP listener def init(self, cluster_ip, user, password, port): cfg.CONF() cluster_url = "rabbit://" + user + ":" + password + "@" + cluster_ip + ":" + port + "/" transport = oslo_messaging.get_notification_transport(cfg.CONF, url=cluster_url) targets = [ oslo_messaging.Target(topic='versioned_notifications'), ] endpoints = [self.__endpoint] # Initialize the notification listener try: self.__amqp_listener = oslo_messaging.get_notification_listener(transport, targets, endpoints, executor='threading') except NotImplementedError as err: LOGGER.error("Failed to initialize the notification listener {}".format(err)) return False LOGGER.debug("Initialized the notification listener {}".format(cluster_url)) return True # Arm the compute event listeners def start_amqp_event_listener(self): # Start the notification handler LOGGER.debug("Started the OpenStack notification handler") self.__amqp_listener.start() # Disarm the compute event listeners def stop_amqp_event_listener(self): LOGGER.debug("Stopping the OpenStack notification handler") if self.__amqp_listener is not None: self.__amqp_listener.stop() I am using this interface from a new process handler function, however, when I invoke the stop_amqp_eent_listener() method, my process hangs. It does not terminate naturally. I verified that the self.__amqp_listener.stop() function is not returning. Is there anything missing in this code? Is there any specific consideration when calling the listener from a new process? Can someone provide a clue? # Stop the worker def stop(self): # Stop the AMQP notification handler self.__amqp_handler.stop_amqp_event_listener() LOGGER.debug("Stopped the worker for {}".format(self.__ops_conn_info.cluster_ip)) /anil.

1 1

[Magnum][Kayobe] Magnum Kubernetes clusters failing to be created (bugs?)
by Tony Pearce 02 Sep '20

02 Sep '20

Hi guys, I hope you are all keeping safe and well at the moment. I am trying to launch Kubernetes clusters into Openstack Train which has been deployed via Kayobe (Kayobe as I understand is a wrapper for kolla-ansible). There have been a few strange issues here and I've struggled to isolate them. These issues started recently after a fresh Openstack deployment some months ago (around February 2020) to give some context. This Openstack is not "live" as I've been trying to get to the bottom of the issues: Issue 1. When trying to launch a cluster we get error "Resource Create Failed: Forbidden: Resources.Kube Masters.Resources[0].Resources.Kube-Master: Only Volume-Backed Servers Are Allowed For Flavors With Zero Disk. " Issue 2. After successfully creating a cluster of a smaller node size, the "resize cluster" is failing (however update the cluster is working). Some background on this specific environment: Deployed via Kayobe, with these components: Cinder, Designate, iscsid, Magnum, Multipathd, neutron provider networks The Cinder component integrates with iSCSI SAN storage using the Nimble driver. This is the only storage. In order to prevent Openstack from allocating Compute node local HDD as instance storage, I have all flavours configured with root disk / ephemeral disk / swap disk = "0MB". This then results in all instance data being stored on the backend Cinder storage appliance. I was able to get a cluster deployed by first creating the template as needed, then when launching the cluster Horizon prompts you for items already there in the template such as number of nodes, node flavour and labels etc. I re-supplied all of the info (as to duplicate it) and then tried creating the cluster. After many many times trying over the course of a few weeks to a few months it was successful. I was then able to work around the issue #2 above to get it increased in size. When looking at the logs for issue #2, it looks like some content is missing in the API but I am not certain. I will include a link to the pastebin below [1]. When trying to resize the cluster, Horizon gives error: "Error: Unable to resize given cluster id: 99693dbf-160a-40e0-9ed4-93f3370367ee". I then searched the controller node /var/log directory for this ID and found "horizon.log [:error] [pid 25] Not Found: /api/container_infra/clusters/99693dbf-160a-40e0-9ed4-93f3370367ee/resize". Going to the Horizon menu "update cluster" allows you to increase the number of nodes and then save/apply the config which does indeed resize the cluster. Regarding issue #1, we've been unable to deploy a cluster in a new project and the error is hinting it relates to the flavours having 0MB disk specified, though this error is new and we've been successful previously with deploying clusters (albeit with the hit-and-miss experiences) using the flavour with 0MB disk as described above. Again I searched for the (stack) ID after the failure, in the logs on the controller and I obtained not much more than the error already seen with Horizon [2]. I was able to create new flavours with root disk = 15GB and then successfully deploy a cluster on the next immediate try. Update cluster from 3 nodes to 6 nodes was also immediately successful. However I see the compute nodes "used" disk space increasing after increasing the cluster size which is an issue as the compute node has very limited HDD capacity (32GB SD card). At this point I also checked 1) previously installed cluster using the 0MB disk flavour and 2) new instances using the 0MB disk flavour. I notice that the previous cluster is having host storage allocated but while the new instance is not having host storage allocated. So the cluster create success is using flavour with disk = 0MB while the result is compute HDD storage being consumed. So with the above, please may I clarify on the following? 1. It seems that 0MB disk flavours may not be supported with magnum now? Could the experts confirm? :) Is there another way that I should be configuring this so that compute node disk is not being consumed (because it is slow and has limited capacity). 2. The issue #1 looks like a bug to me, is it known? If not, is this mail enough to get it realised? Pastebin links as mentioned [1] http://paste.openstack.org/show/797316/ [2] http://paste.openstack.org/show/797318/ Many thanks, Regards, Tony Pearce

2 4

[ptg][glance] Wallaby PTG planning etherpad
by Abhishek Kekane 02 Sep '20

02 Sep '20

Hi All, The Wallaby PTG will be held next month from Oct 26 to Oct 30. I have booked some time slots in the ethercalc[0], however these are not fixed, if the booked time does not suit your time, please let me know. I have also created a planning etherpad [1] where you can add your topics for discussion during PTG. [0]https://ethercalc.openstack.org/7xp2pcbh1ncb. [1] https://etherpad.opendev.org/p/Glance-Wallaby-PTG-planning Thank you, Abhishek Kekane

1 0

Issue with heat and magnum
by Alexander Dibbo - UKRI STFC 02 Sep '20

02 Sep '20

Hi, I am having an issue with magnum creating clusters when I have multiple active heat-engine daemons running. I get the following error in the heat engine logs: 2020-08-14 10:36:30.237 598383 INFO heat.engine.resource [req-a2c862eb-370c-4e91-a2c6-dca32c7872ce - - - - -] signal SoftwareDeployment "master_config_deployment" [67ba9ce2-aba5-4c15-a7ea -6b774659a0e2] Stack "kubernetes-test-26-3uzjqqob47fh-kube_masters-mhctjio2b4gh-0-pbhumflm5mn5" [dc66e4d9-0c9b-4b18-a2c6-dd9724fa51a9] : Authentication cannot be scoped to multiple target s. Pick one of: project, domain, trust or unscoped 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource Traceback (most recent call last): 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 2462, in _handle_signal 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource signal_result = self.handle_signal(details) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/software_deployment.py", line 514, in handle_signal 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource timeutils.utcnow().isoformat()) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 788, in signal_software_deployment 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource version='1.6') 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/rpc/client.py", line 89, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return client.call(ctxt, method, **kwargs) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 165, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource msg_ctxt = self.serializer.serialize_context(ctxt) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/messaging.py", line 46, in serialize_context 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource _context = ctxt.to_dict() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 185, in to_dict 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource 'roles': self.roles, 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 315, in roles 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource self._load_keystone_data() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 292, in wrapped_f 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return self.call(f, *args, **kw) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 358, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource do = self.iter(retry_state=retry_state) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 319, in iter 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return fut.result() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 422, in result 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return self.__get_result() 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 361, in call 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource result = fn(*args, **kwargs) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 306, in _load_keystone_data 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource auth_ref = self.auth_plugin.get_access(self.keystone_session) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource self.auth_ref = self.get_auth_ref(session) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource return self._plugin.get_auth_ref(session, **kwargs) 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 144, in get_auth_ref 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource message='Authentication cannot be scoped to multiple' 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource AuthorizationFailure: Authentication cannot be scoped to multiple targets. Pick one of: project, domain, trust or unscoped 2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service [req-a2c862eb-370c-4e91-a2c6-dca32c7872ce - - - - -] Unhandled error in asynchronous task: ResourceFailure: AuthorizationFailure: resources.master_config_deployment: Authentication cannot be scoped to multiple targets. Pick one of: project, domain, trust or unscoped 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service Traceback (most recent call last): 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 132, in log_exceptions 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service gt.wait() 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 181, in wait 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service return self._exit_event.wait() 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 132, in wait 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service current.throw(*self._exc) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 221, in main 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service result = function(*args, **kwargs) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 123, in _start_with_trace 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service return func(*args, **kwargs) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 1871, in _resource_signal 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service needs_metadata_updates = rsrc.signal(details, need_check) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 2500, in signal 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service self._handle_signal(details) 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 2480, in _handle_signal 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service raise failure 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service ResourceFailure: AuthorizationFailure: resources.master_config_deployment: Authentication cannot be scoped to multiple targets. Pi ck one of: project, domain, trust or unscoped 2020-08-14 10:36:30.890 598383 ERROR heat.engine.service Each of the individual heat-engine daemons create magnum clusters correctly when they are the only ones online. Attached are the heat and magnum config files. Any ideas where to look would be appreciated? Regards Alexander Dibbo - Cloud Architect / Cloud Operations Group Leader For STFC Cloud Documentation visit https://stfc-cloud-docs.readthedocs.io<https://stfc-cloud-docs.readthedocs.io/> To raise a support ticket with the cloud team please email cloud-support(a)gridpp.rl.ac.uk<mailto:cloud-support@gridpp.rl.ac.uk> To receive notifications about the service please subscribe to our mailing list at: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD To receive fast notifications or to discuss usage of the cloud please join our Slack: https://stfc-cloud.slack.com/ This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. Opinions, conclusions or other information in this message and attachments that are not related directly to UKRI business are solely those of the author and do not represent the views of UKRI.

2 1

[ptg][cyborg] Cyborg Wallaby PTG schedule and resources
by yumeng bao 02 Sep '20

02 Sep '20

Hi team, The next virtual Wallaby PTG will be held next month from Oct 26 to Oct 30. I have booked the same time slots in the ethercalc[0] as our last PTG did. However, these are not fixed, if the booked time does not suit your time, Please don't hesitate to add your availability to the doodle[1]. BTW, PTG Topics are open to collect and you can add to the etherpad[2]. [0]https://ethercalc.openstack.org/7xp2pcbh1ncb. [1]https://doodle.com/poll/mrudbrn87vh48ayq [2]https://etherpad.opendev.org/p/cyborg-wallaby-goals Regards， Yumeng

1 0

[scientific-sig] SIG Meeting: Ironic kexec, CANOPIE-HPC and the PTG
by Stig Telfer 01 Sep '20

01 Sep '20

Hi All - We have a Scientific SIG meeting on IRC channel #openstack-meeting at 2100 UTC (about 45 minutes). Everyone is welcome. Agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_September_1st_20… Today we'd like to talk about supporting the development of kexec support for Ironic deployments. Also some up-coming CFPs and PTG participation. Cheers, Stig

1 0

[nova] providing a local disk to a Nova instance
by Thomas Goirand 01 Sep '20

01 Sep '20

Hi Nova team! tl;dr: we would like to contribute giving instances access to physical block devices directly on the compute hosts. Would this be accepted? Longer version: About 3 or 4 years ago, someone wrote a spec, so we'd be able to provide a local disk of a compute, directly to a VM to use. This was then rejected, because at the time, Cinder had the blockdevice drive, which was more or less achieving the same thing. Unfortunately, because nobody was maintaining the blockdevice driver in Cinder, and because there was no CI that could test it, the driver got removed. We've investigated how we could otherwise implement it, and one solution would be to use Cinder, but then we'd be going through an iSCSI export, which would drastically reduce performances. Another solution would be to manage KVM instances by hand, not touching anything to libvirt and/or OpenVSwitch, but then we would loose the ease of using the Nova API, so we would prefer to avoid this direction. So we (ie: employees in my company) need to ask the Nova team: would you consider a spec to do what was rejected before, since there's now no other good enough alternative? Our current goal is to be able to provide a disk directly to a VM, so that we could build Ceph clusters with an hyper-converged model (ie: storage hosted on the compute nodes). In this model, we wouldn't need live-migration of a VM with an attached physical block device (though the feature could be added on a later stage). Before we start investigating how this can be done, I need to know if this has at least some chances to be accepted or not. If there is, then we'll probably start an experimental patch locally, then write a spec to properly start this project. So please let us know. Cheers, Thomas Goirand (zigo)

4 3

[neutron] Bug deputy report August 24 - 30
by Rodolfo Alonso Hernandez 01 Sep '20

01 Sep '20

Hello: This is the list of bugs in the period from August 24 to August 30. Critical: (none) High: - https://bugs.launchpad.net/neutron/+bug/1892861: [neutron-tempest-plugin] If paramiko SSH client connection fails because of authentication, cannot reconnect (unassigned) - https://bugs.launchpad.net/neutron/+bug/1893314: set neutron quota not take effect (ZhouHeng, https://review.opendev.org/#/c/748578/) Medium: - https://bugs.launchpad.net/neutron/+bug/1892496: 500 on SG deletion: Cannot delete or update a parent row (unassigned) - https://bugs.launchpad.net/neutron/+bug/1888878: Data not present in OVN IDL when requested (fix released) - https://bugs.launchpad.net/neutron/+bug/1892846: AttributeError in l3-agent changing HA router to non-HA (Slawek, https://review.opendev.org/#/c/747922/) Low: - https://bugs.launchpad.net/neutron/+bug/1892741: neutron-sriov-nic-agent cannot be disabled (unassigned) - https://bugs.launchpad.net/neutron/+bug/1893188: [tempest] Randomize subnet CIDR to avoid test clashes (Lajos, https://review.opendev.org/#/c/749041/) Wishlist: - https://bugs.launchpad.net/neutron/+bug/1893625: Implement router gateway IP QoS in OVN backend (zhanghao, https://review.opendev.org/#/c/749012/) Incomplete: - https://bugs.launchpad.net/neutron/+bug/1893015: ping with large package size fails Regards.

3 2

2020 Virtual Summit: Forum Submissions Now Accepted
by Jimmy McArthur 01 Sep '20

01 Sep '20

Hello Everyone! We are now accepting Forum [1] submissions for the 2020 Virtual Open Infrastructure Summit [2]. Please submit your ideas through the Summit CFP tool [3] through September 14th. Don't forget to put your brainstorming etherpad up on the Shanghai Forum page [4]. This is not a classic conference track with speakers and presentations. OSF community members (participants in development teams, operators, working groups, SIGs, and other interested individuals) discuss the topics they want to cover and get alignment on and we welcome your participation. The Forum is your opportunity to help shape the development of future project releases. More information about the Forum [1]. The timeline for submissions is as follows: Aug 31st | Formal topic submission tool opens: https://cfp.openstack.org. Sep 14th | Deadline for proposing Forum topics. Scheduling committee meeting to make draft agenda. Sep 21st | Draft Forum schedule published. Crowd sourced session conflict detection. Forum promotion begins. Sept 28th | Forum schedule final Oct 19th | Forum begins! If you have questions or concerns, please reach out to speakersupport(a)openstack.org. Cheers, Jimmy [1] https://wiki.openstack.org/wiki/Forum [2] https://www.openstack.org/summit/2020/ [3] https://cfp.openstack.org [4]https://wiki.openstack.org/wiki/Forum/Virtual2020

1 1

[tripleo][ussuri][undercloud][tripleo_rabbitmq] do not start after reboot
by Ruslanas Gžibovskis 01 Sep '20

01 Sep '20

Hi all, I think this issue might be known to RHOSP16 [1], but it is same in RDO - ussuri with centos8. if anything is needed, I could try to help/test. my workaround is to run openstack undercloud install --force-stack-update [1] https://access.redhat.com/solutions/5269241 -- Ruslanas Gžibovskis +370 6030 7030

1 0

[horizon] horizon-integration-tests job is broken
by Ivan Kolodyazhny 01 Sep '20

01 Sep '20

Hi team, Please, do not trigger recheck if horizon-integration-tests failed like [1]: ______________ TestDashboardHelp.test_dashboard_help_redirection _______________ 'NoneType' object is not iterable While I'm trying to figure out what is happening there [2], any help with troubleshooting is welcome. [1] https://51bb980dc10c72928109-9873e0e5415ff38d9f1a5cc3b1681b19.ssl.cf1.rackc… [2] https://review.opendev.org/#/c/749011 Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 1

[ops] Restructuring OSOPS tools
by Sean McGinnis 31 Aug '20

31 Aug '20

Hello everyone, We recently expanded the scope of the Ops Docs SIG to also include any ops tooling. I think it's now time to move on to the next step of actually getting some of the old tooling in place and organized how we want it. We have several semi-abandoned repos from back when there was more work being done on ops tooling. During the great rebranding, those all were moved under the x/ namespace: https://opendev.org/x/?tab=&sort=recentupdate&q=osops Since these are now owned by an official SIG, we can move this content back under the openstack/ namespace. That should help increase visibility somewhat, and make things look a little more official. It will also allow contributors to tooling to get recognition for contributing to an import part of the OpenStack ecosystem. I do think it's can be a little more difficult to find things spread out over several repos though. For simplicity with finding tooling, as well as watching for reviews and helping with overall maintenance, I would like to move all of these under a common openstack/osops. Under that repo, we can then have a folder structure with tools/logging, tools/monitoring, etc. Then with everything in one place, we can have docs published in one place that helps find everything and easily links between tools. We can also capture some metadata about the tools, and use that to reflect their state in those docs. Please let me know if there are any objects to this plan. Otherwise, I will start cleaning things up and getting it staged in a new repo to be imported as an official repo owned by the SIG. Thanks! Sean

3 2

[TripleO][Ussuri] image prepare, cannot download containers image prepare recently
by Ruslanas Gžibovskis 31 Aug '20

31 Aug '20

Hi all, I have noticed, that recently my undercloud is not able to download images [0]. I have provided newly generated containers-prepare-parameter.yaml and outputs from container image prepare providing --verbose and later beginning of --debug (in the end) [0] were there any changes? As "openstack tripleo container image prepare default --output-env-file containers-prepare-parameter.yaml --local-push-destination" have prepared a bit different file, compared what was previously: NEW # namespace: docker.io/tripleou VS namespace: docker.io/tripleomaster # OLD [0] - http://paste.openstack.org/show/rBCNAQJBEe9y7CKyi9aG/ -- Ruslanas Gžibovskis +370 6030 7030

2 1

[neutron] Team meeting - Tuesday 01.09.2020
by Slawek Kaplonski 31 Aug '20

31 Aug '20

Hi, I will be on PTO on Tuesday and I will not be able to run our team meeting. So lets cancel it this week and see You all on meeting next week. For this week there are 3 important things from me: * please check doodle: https://doodle.com/poll/2ppmnua2nuva5nyp and put there time slots which works the best for You for the PTG in October - please do that this week as next week I need to book some slots for us, * as we are close to the Victoria-3 milestone (next week) which is also feature freeze week, please focus now on reviewin patches for BPs targeted for this cycle: https://wiki.openstack.org/wiki/Network/Meetings#Blueprints * There is new bug deputy role starting this week. Please check new schedule at https://wiki.openstack.org/wiki/Network/Meetings#Bug_deputy and let me know if it don't works for You. That's all from me for this week. See You all online :) -- Slawek Kaplonski Principal software engineer Red Hat

1 0

[tc] weekly update
by Mohammed Naser 31 Aug '20

31 Aug '20

Hi everyone, Here’s an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. We've also included a few references to some important mailing list threads that you should check out. # Patches ## Open Reviews - Move ansible-role-XXX-hsm projects to Barbican team https://review.opendev.org/748027 - Retire the devstack-plugin-zmq project https://review.opendev.org/748731 - Retire devstack-plugin-pika project https://review.opendev.org/748730 - Add openstack-ansible/os_senlin role https://review.opendev.org/748677 - Drop all exceptions for legacy validation https://review.opendev.org/745403 - Add openstack-helm-releases to openstack-helm https://review.opendev.org/748302 - Add assert:supports-standalone. https://review.opendev.org/722399 ## Project Updates - Add etcd3gw to Oslo https://review.opendev.org/747188 ## General Changes - Update and simplify comparison of working groups https://review.opendev.org/746763 - Move towards dual office hours in diff TZ https://review.opendev.org/746167 ## Abandoned Changes - Drop requirement of 1/3 positive TC votes to land https://review.opendev.org/746711 - Move towards single office hour https://review.opendev.org/745200 # Email Threads - vPTG October 2020 Signup: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016497.h… Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 0

Setuptools 48 and Devstack Failures
by Clark Boylan 31 Aug '20

31 Aug '20

Hello, Setuptools has made a new version 48 release. This appears to be causing problems for devstack because `pip install -e $PACKAGE_PATH` installs commands to /usr/bin and not /usr/local/bin on Ubuntu as it did in the past. `pip install $PACKAGE_PATH` continues to install to /usr/local/bin as expected. Devstack is failing because keystone-manage cannot currently be found at the specific /usr/local/bin/ path. Potential workarounds for this include not using `pip install -e` or relying on $PATH to find the commands rather than specifying rooted paths to them. I'll defer to the QA team on how they want to address this. While we can have devstack install an older setuptools version as well, generally this is not considered to be a good idea because anyone doing pip installs outside of devstack may get the newer behavior. It is actually important for us to try and keep up with setuptools changes as a result. Fungi indicated that setuptools expected this to be a bumpy upgrade. I'm not sure if they would consider `pip install -e` and `pip install` installing to different paths as a bug, and if they did which behavior is correct. It would probably be a good idea to file a bug upstream if we debug this further. Clark

5 6

[octavia] usage of SELECT .. FOR UPDATE
by Mohammed Naser 30 Aug '20

30 Aug '20

Hi everyone, We're being particularly hit hard across different deployments where Octavia has several SELECT .. FOR UPDATE queries which are causing load balancers to fail to provision properly. - spare_pools: This usually hits on rolling restarts of o-housekeeping as they all seem to try to capture a lock -- https://github.com/openstack/octavia/blob/73fbc05386b512aa1dd86a0ed6e8455cc… - quota: This hits when provisioning a lot of load balancers in parallel. For example in cases when using Heat -- https://github.com/openstack/octavia/blob/bf3d5372b9fc670ecd08339fa989c9b73… These hurt quite a lot in a busy deployment and result in a poor user experience unfortunately. We're trying to off-load Octavia to it's own database server but that is more of a "throw power at the problem" solution. I can imagine that we can probably likely look into a better/cleaner alternative that avoids this entirely? I'm happy to try and push for some of this work on our side. Thanks, Mohammed -- Mohammed Naser VEXXHOST, Inc.

2 1

Openstack zun ui
by mahdi abbasi 30 Aug '20

30 Aug '20

Hi, I want install zun ui, first i installed horizon with package and then install zun ui with pip and then python horizon_path/manage.py collectstatic But i receive error in httpd finaly. Error: ImportError: No module named zun_ui Please help me Best Regards Mahdi

2 4

[all] READMEs of zuul roles not rendered properly - missing content
by Martin Kopec 30 Aug '20

30 Aug '20

Hello everyone, I've noticed that READMEs of zuul roles within openstack projects are not rendered properly on opendev.org - ".. zuul:rolevar::" syntax seems to be the problem. Although it's rendered well on github.com, see f.e. [1] [2]. I wonder if there were some changes in the supported README syntax. Also the ".. zuul:rolevar::" syntax throws errors on online rst formatters I was testing on, however, it's rendered fine by md online formatters - maybe opendev.org is more rst strict in case of .rst files than github? Any ideas? [1] https://opendev.org/openstack/tempest/src/branch/master/roles/run-tempest [2] https://github.com/openstack/tempest/tree/master/roles/run-tempest Thanks, -- Martin Kopec Quality Engineer Red Hat EMEA <https://www.redhat.com> <https://red.ht/sig>

4 4

[tc][telemetry][gnocchi] The future of Gnocchi in OpenStack
by Adrian Turjak 28 Aug '20

28 Aug '20

Hey OpenStackers, We're currently in the process of discussing what to do with OpenStack's reliance on Gnocchi, and at present it is looking like we are most likely to just fork it back under a new name (currently Farfalle to stick with the pasta theme). The discussion is mostly happening here: https://review.opendev.org/#/c/744592/ But for those running Gnocchi in prod, this is likely something you may want to know about and we'd like to hear from you. A bit of history: Gnocchi started off as a new backend for Ceilometer in OpenStack, and eventually become the defacto API for telemetry samples when that was removed from Ceilometer (as backed by MongoDB). Gnocchi was eventually spun off outside of OpenStack, but still essentially remained our API for telemetry despite not being an official part of OpenStack anymore. Since then the development around it seems to have stalled, with pull requests left unreviewed, CI broken, and even the domain for the docs lapsing once. They have essentially said the project is unmaintained themselves: https://github.com/gnocchixyz/gnocchi/issues/1049 Given that OpenStack telemetry relies on it, we needed to decide what to do. We tried talking to the devs which spun it off outside of OpenStack, but they seem disinclined to interact with the OpenStack community, or move the project back to our infra/governance despite OpenStack looking like the only consumers of Gnocchi as a project. We want to find a solution, and the feeling is that they don't. So we've opted to fork it back and now the discussion is how to approach that fork. The OpenStack community doesn't want to maintain a time series database, but our telemetry API is part of it. We are putting it under non-OpenStack namespace to start, but we need to decide what the long term place for it should be. Do we want to make it an official project again? Do we keep it just as an API and drop the time series DB part for another DB? Do we build a new API back into Ceilometer and switch to a different backend like InfluxDB? We don't know yet, and we want some input from people who use the service so we can hopefully work with OpenStack telemetry as a whole and figure out what the long term picture is. If Gnocchi matters to you at all, or you use it, we want to hear from you. Cheers, Adrian Turjak

3 3

[release] Release countdown for week R-6 Aug 31 - Sept 4
by Sean McGinnis 28 Aug '20

28 Aug '20

Development Focus ----------------- Work on libraries should be wrapping up, in preparation for the various library-related deadlines coming up. Now is a good time to make decisions on deferring feature work to the next development cycle in order to be able to focus on finishing already-started feature work. General Information ------------------- We are now getting close to the end of the cycle, and will be gradually freezing feature work on the various deliverables that make up the OpenStack release. This coming week is the deadline for general libraries (except client libraries): their last feature release needs to happen before "Non-client library freeze" on September 3. Only bugfix releases will be allowed beyond this point. When requesting those library releases, you can also include the stable/victoria branching request with the review (as an example, see the "branches" section here: https://opendev.org/openstack/releases/src/branch/master/deliverables/pike/… In the next weeks we will have deadlines for: * Client libraries (think python-*client libraries), which need to have their last feature release before "Client library freeze" (Sept 10) * Deliverables following a cycle-with-rc model (that would be most services), which observe a Feature freeze on that same date, Sept 10. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. As we are getting to the point of creating stable/victoria branches, this would be a good point for teams to review membership in their victoria-stable-maint groups. Once the stable/victoria branches are cut for a repo, the ability to approve any necessary backports into those branches for Victoria will be limited to the members of that stable team. If there are any questions about stable policy or stable team membership, please reach out in the #openstack-stable channel. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: September 3 (R-6 week) Client library freeze: September 10 (R-5 week) Victoria-3 milestone: September 10 (R-5 week) Cycle Highlights Due: September 10 (R-5 week) Victoria release: October 14

1 0

[all][goals] Switch legacy Zuul jobs to native - update #3
by Luigi Toscano 28 Aug '20

28 Aug '20

Hi, it's time for another status report on this goal. A lot of reviews have been merged in the past 10 days, and several projects are not on the list anymore. This is a very good news, but we still have some work to complete. Please keep pushing! Status ====== The number of the project with legacy jobs is now limited, so I'm going to explain the status in more details for each of them. The links to the patches can be found in the etherpad [3] (see below for the links). cinder ------ There is just one test left, and it is definitely tricky, because it implements a cycle of "change tempest configuration"/"run tempest"/"repeat", which is not the usual pattern. In the worst case I will "port" by adding a simple bash wrapper, but I'd like to have a clean ansible solution. designate --------- A patch for the only legacy job is under review after some forth and back, but there are some open questions. heat ------ Only one legacy job left, the heat cores are aware of it. infra ----- Only one devstack-gate job in the os-loganalyze repository, which should be probably retired. There are 2 other legacy jobs, but not devstack-gate, so less urgent. ironic ------ There has been an open review with the full port of the last legacy job, but it is failing. As it is has been failing even before the porting, the patch could be probably merged as it is. karbor ------ A patch for the only legacy job is under review, but it still has some issues. manila ------ There is only one legacy-base job (not devstack-gate), so less urgent, but there is a patch for it. monasca ------- One job in the monasca-transform repository, which is most likely due for a retirement. There are 3 legacy (non devstack-gate) jobs in other repositories. murano ------ There are two legacy jobs left. I'm not sure whether murano-apps-refstackclient-unittest is still needed. murano-dashboard-sanity-check is a bit tricky, the tests still use nose and the corresponding code in horizon has seen several changes. neutron ------- There are three types of legacy jobs: * all jobs in networking-midonet, whose retirement is under discussion, but the final decision is not clear, so a porting may be needed anyway: * two grenade jobs are being worked on; * the remaining legacy job could be maybe dropped. nova ---- The team is trying to port the two legacy job left with some refactoring, but it may require some effort yet. oslo ---- Only one legacy job left, but it is part of the soon-to-be-retired devstack-plugin-zmq repository. senlin ------ A patch for the only legacy job has been proposed and it is working, needs reviews. trove ----- The trove-grenade job should be ported, but on the other hand, trove has no grenade plugin. At this point it is unlikely to be implemented before Victoria, so maybe the job can be dropped for now. zaqar ----- A few patches have been proposed and working. One of them is failing (python-zaqarclient) but it does a bit more than a simple porting, so it may be simply changed to do exactly what the old job was doing (input needed). References ========== [1] the goal: https://governance.openstack.org/tc/goals/selected/victoria/native-zuulv3-j… [2] the up-to-date Zuul v3 porting guide: https://docs.openstack.org/project-team-guide/zuulv3.html [3] the etherpad which tracks the current status: https://etherpad.opendev.org/p/goal-victoria-native-zuulv3-migration [4] the previous reports: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016058.html http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016561.h… Ciao -- Luigi

1 0

[neutron][gate] verbose q-svc log files and e-r indexing
by melanie witt 28 Aug '20

28 Aug '20

Hi all, Recently we've noticed elastic search indexing is behind 115 hours [1] and we looked for abnormally large log files being generated in the gate. We found that the q-svc log is very large, one example being 71.6M [2]. There is a lot of Time-Cost profiling output in the log, like this: Aug 17 14:22:23.210076 ubuntu-bionic-ovh-bhs1-0019298855 neutron-server[5168]: DEBUG neutron_lib.utils.helpers [req-75719db1-4abf-4500-bb0a-6d24e82cd4fd req-d88e7052-7da9-4bc9-8b35-5730ae76dcad service neutron] Time-cost: call 48e628cc-8c3a-408d-a36f-b219524480e0 function apply_funcs start {{(pid=5554) wrapper /usr/local/lib/python3.6/dist-packages/neutron_lib/utils/helpers.py:218}} We saw that there was a recent-ish change to remove some of the profiling output [3] but it was only for the get_objects method. Looking at the total number of lines in the file vs the number of lines without apply_funcs Time-Cost output: $ wc -l screen-q-svc.txt 186387 screen-q-svc.txt $ grep -v "function apply_funcs" screen-q-svc.txt|wc -l 102593 Would it be possible to remove this profiling output from the gate log to give elastic search indexing a better chance at keeping up? Or is there something else I've missed that could be made less verbose in the logging? Thanks for your help. Cheers, -melanie [1] http://status.openstack.org/elastic-recheck [2] https://b6ba3b9af8fd7de57099-18aa39cea11f738aa67ebd6bc9fb5e4c.ssl.cf2.rackc… [3] https://review.opendev.org/741540

3 5

[all][tc][policy] Progress report of consistent and secure default policies pop-up team
by Ghanshyam Mann 28 Aug '20

28 Aug '20

Hello Everyone, This is a regular update on progress in 'Consistent and Secure Default Policies Popup Team'. We will try to make it a monthly report form now onwards. Progress so far: ============ * Popup team meet twice in a month and discuss and work on progress and pre-work to do. - https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… * Pre-work to provide a smooth migration path to the new policy <here we will add any pre-work we need to do before more project start moving towards new policy> ** Migrate Default Policy Format from JSON to YAML - This involves oslo side + each project side works. - oslo side work to provide tool and utils method are merged (one patch is in gate). - The new tool 'oslopolicy-convert-json-to-yaml' is available now to convert your existing JSON formatted policy file to YAML formatted in a backward-compatible way. - I have started to do it in Nova (need to update the patch though) to give example work for other projects: https://review.opendev.org/#/c/748059/ - all work is tracked here: https://review.opendev.org/#/q/topic:bp/policy-json-to-yaml+(status:open+OR… ** Improving documentation about target resources (oslo.policy) - https://bugs.launchpad.net/oslo.policy/+bug/1886857 - raildo pushed the patch which is under review: https://review.opendev.org/#/c/743318/ * Team Progress: (list of a team interested or have volunteer to work) ** Keystone (COMPLETED; use as a reference) ** Nova (COMPLETED; use as a reference) - All APIs except deprecated APIs were done in the Ussuri cycle and deprecated APIs also done now. ** Cyborg (in-progress) - Spec is merged, code under review. ** Barbican (not started) ** Neutron (not started) ** Cinder (not started) ** Manila (not started) Why This Is Important ================= (I have copied it from Colleen email which is nicely written) Separating system, domain, and project-scope APIs and providing meaningful default roles is critical to facilitating secure cloud deployments and to fulfilling OpenStack's vision as a fully self-service infrastructure provider[1]. Until all projects have completed this policy migration, the "reader" role that exists in keystone is dangerously misleading, and the `[oslo_policy]/enforce_scope` option has limited usefulness as long as projects lack uniformity in how an administrator can use scoped APIs. How You Can Help ================ Contributor: - You can help by starting the work in your (or any other you would like to help) project and attend popup team meeting in case of any question, review request etc. Cloud operator: - Please help review the proposed policy rule changes to sanity-check the new scope and role defaults. - Migrate your JSON formatted policy file to YAML JSON formatted file can be problematic in the various way as described here[2]. You can use 'oslopolicy-convert-json-to-yaml' tool [3] to convert your existing JSON formatted policy file to YAML formatted in a backward-compatible way. [1] https://governance.openstack.org/tc/reference/technical-vision.html#self-se… [2] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/policy-json… [3] https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-t… -gmann & raildo

1 0

Horizon installation problem
by mahdi abbasi 28 Aug '20

28 Aug '20

Hi openstack development team, When i want install horizon with pip3 i recieve and error: Could not satisfy constraints for horizon: installation from path or url cannot be constrained to a version. Please help me Best regards Mahdi

2 1