- openstack-discuss - lists.openstack.org

[neutron] Next step for BGP routed networks over segmented provider infrastructure segments
by Thomas Goirand 06 Jan '21

06 Jan '21

Hi Ryan, and all of the Neutron team, Today, I'm happy to let you know that I've been able to finish the patch and that it's merged: https://review.opendev.org/c/openstack/neutron/+/669395 I also managed to add some docs to Neutron about it: https://docs.openstack.org/neutron/latest/admin/config-bgp-floating-ip-over… We've used it in a pre-production environment, and it just works as expected, it's kind of great. However, there's some feature gaps that would need to be addressed. Namely: - external-gateway of routers aren't advertized - we can't do direct attach of public IPs to VMs - I failed adding IPv6 dual stack to this setup Let me go into more details for each of these 3 points. 1/ No BGP advertizing for the router default gateways When doing: openstack router set --external-gateway <provider-net> <router> we then get this type of port: # openstack port show <UUID> -c binding_vif_details -c binding_vif_type +---------------------+-------------------------------------------------------------------------------------------------------------+ | Field | Value | +---------------------+-------------------------------------------------------------------------------------------------------------+ | binding_vif_details | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='True', port_filter='True' | | binding_vif_type | ovs | +---------------------+-------------------------------------------------------------------------------------------------------------+ which doesn't match the type of port we have for floating IPs: # openstack port show <UUID> -c binding_vif_details -c binding_vif_type +---------------------+---------+ | Field | Value | +---------------------+---------+ | binding_vif_details | | | binding_vif_type | unbound | +---------------------+---------+ and then, the next HOP for the router gateway isn't advertized over BGP. Do you know how we could get neutron-dynamic-routing to do that advertizing, with the next HOP on the network node(s)? Where should that code be patch? Inside Neutron, or in neutron-dynamic-routing? Is this really related to the port type as I've showed above? 2/ No direct attach to VM ports We can't attach a port with an IP network:routed directly to a VM. I tried to add the subnet type "compute:nova" to the floating IP subnet, but that didn't do it: Neutron refuses to attach the port to a VM. Do you know why? How and what and where should we patch Neutron to fix this? 3/ IPv6 dual stack I tried to setup a dual-stack network, and failed. How should this be done? Should we add v6 subnets to segments and one subnet with the type --service-type 'network:router_gateway' as well? This is what I tried but it didn't work for me. Should tenants create their own v6 subnet out of the v6 subnet pool I provisioned as admin? Cheers, Thomas Goirand (zigo) P.S: Please keep my Infomaniak colleagues as Cc.

1 0

failed to launch openstack instance when integrating with odl
by mohammad mighani 05 Jan '21

05 Jan '21

Hi everyone I tried to integrate Openstack train with Opendaylight magnesium and my references were [1] and [2] but after doing all the steps, the instance failed to launch. I changed port_binding_controller in [ml2_odl] section in the ml2_conf.ini file from pseudo-agentdb-binding to legacy-port-binding and then the instance launched but the status of router interfaces was still down. I have a controller node and a compute node. and Opendaylight runs on the controller node. nova-compute.log: INFO nova.virt.libvirt.driver [-] [instance: 975fa79e-6567-4385-be87-9d12a8eb3e94] Instance destroyed successfully. 2021-01-02 12:33:23.383 25919 ERROR vif_plug_ovs.ovsdb.impl_vsctl [req-a0a7ebf0-7e63-4c60-a8d2-07c05f1aa4f4 04c7685a2166481a9ace54eb5e71f6e5 ca28ee1038254649ad133d5f09f7a186 - default default] Unable to execute ['ovs-vsctl', '--timeout=120', '--oneline', '--format=json', '--db=tcp: 127.0.0.1:6640', '--', '--if-exists', 'del-port', 'br-int', 'tap50eb0b68-a4']. Exception: Unexpected error while running command. Command: ovs-vsctl --timeout=120 --oneline --format=json --db=tcp: 127.0.0.1:6640 -- --if-exists del-port br-int tap50eb0b68-a4 Exit code: 1 Stdout: '' Stderr: 'ovs-vsctl: tcp:127.0.0.1:6640: database connection failed (Connection refused)\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. controller ovs: Manager "tcp:192.168.222.48:6640" is_connected: true Bridge br-int Controller "tcp:192.168.222.48:6653" is_connected: true fail_mode: secure Port tune52c5c73a50 Interface tune52c5c73a50 type: vxlan options: {key=flow, local_ip="10.0.0.31", remote_ip="10.0.0.11"} Port br-int Interface br-int type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port ens160 Interface ens160 ovs_version: "2.13.1" compute ovs: Manager "tcp:192.168.222.48:6640" is_connected: true Bridge br-int Controller "tcp:192.168.222.48:6653" is_connected: true fail_mode: secure Port br-int Interface br-int type: internal Port tun34b3712d975 Interface tun34b3712d975 type: vxlan options: {key=flow, local_ip="10.0.0.11", remote_ip="10.0.0.11"} Port tund5123ce5b8a Interface tund5123ce5b8a type: vxlan options: {key=flow, local_ip="10.0.0.11", remote_ip="10.0.0.31"} ovs_version: "2.13.1"

1 0

[ironic] inspector auto-discovery disabled by default in bifrost
by Jacob Anders 05 Jan '21

05 Jan '21

Hi Ironicers, The Ironic team recently decided to change the default inspector auto-discovery behavior in bifrost. In the past, this feature was enabled by default. However, this was causing issues in deployments where the operators had no intention of using auto-discovery but the IPA somehow booted anyway, causing port conflicts and creating failure modes which were particularly tricky to debug. For this reason, inspector auto-discovery has been disabled by default with this change which merged yesterday: https://review.opendev.org/c/openstack/bifrost/+/762998 This is to make sure that the operators utilising inspector auto-discovery in bifrost are aware of this change and can re-enable inspector discovery if desired. Best regards, Jacob

1 0

[Nova]ask for help: how to plug vif to ovs when ovs is not together with nova-compute
by 付志华 05 Jan '21

05 Jan '21

hello， I encountered some problems in deploying the smart network interface card（SmartNIC）where Nova-compute is not together with OVS. For example, OVS is deployed on the SmartNIC and Nova-compute is on the Host(Compute Node), how does Nova plug the virtual interface of VM to OVS bridge? Does the latest version of openstack support this? Thanks.

2 1

[keystone] No meeting Jan 5
by Nikolla, Kristi 05 Jan '21

05 Jan '21

Hi all, There will be no keystone IRC meeting on Jan 5. Best, Kristi Nikolla

1 0

[tripleo] Package dependencies, podman, dnf modules
by Cédric Jeanneret 05 Jan '21

05 Jan '21

Hello there, Since we introduced Podman instead of Docker in tripleo, we are running after the right version. For now, the version we should use is provided by a specific container-tools module stream (container-tools:2.0). The default stream being container-tools:rhel8, we might end with a deployed undercloud running the wrong podman version, leading to some nice issues, especially in a SELinux-enabled environment. Currently, the main dependency tree is: python3-tripleoclient requires podman That's mostly all. While we can, of course, edit the python3-tripleoclient spec file in order to pin the precise podman version (or, at least, set upper and lower constraints) in order to target the version provided by the right stream, it's not ideal: - package install error will lead to more questions (since ppl might skip doc for #reason) - if we change the target version, we'll need to ensure it's present in a stream (and, well, update the doc if we need to switch the stream) - and probably some other reasons. Since we can NOT depend on a specific stream in rpm (for instance, we can't put "Requires @container-tools:2.0/podman" like we might in ansible), we're a bit stuck. An ugly hack is tested in the spec file[1], using a bash thing in order to check the activated stream, but this is not ideal either, especially since it makes the whole package building/checking fail during the "mock" stage. In order to make it pass, we'd need to modify the whole RDO in order to take into account stream switching. This isn't impossible, but it leads to other concerns, especially when it comes to "hey we need to switch the stream for <module>". And I'm pretty sure no one will want to dig into it, for good reasons ;). This leads to a last possibility: drop the "Requires podman" from tripleo dependency tree, and rely on tripleo-ansible to actually enable the correct module, and install podman. This means podman will be installed during the undercloud deploy, for instance as a host_prep_tasks (step_0). While I'm not that happy with this idea, it's probably not as bad as the other hacks we've tested so far, and would, at last, prevent once for all the usual "it's not working" due to a wrong podman version (and, yes, we get plenty of them, especially in the OSP version). What do you think? Any other way of solving this issue? Remember, we're talking about "ensuring we get the right version, coming from a specific stream" and, especially, "ensure operator can't install the wrong one" (if they don't follow the doc, of if they are using automation that doesn't check actual requirements in the doc"... Thank you for your thoughts! Cheers, C. [1] https://review.rdoproject.org/r/#/c/31411/ -- Cédric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/

4 6

[qa][tempest] Update language in tempest code base
by Arx Cruz 05 Jan '21

05 Jan '21

Hello, I would like to start a discussion regarding the topic. At this moment in time we have an opportunity to be a more open and inclusive project by eliminating outdated naming conventions from tempest codebase, such as blacklist, whitelist. We should take the opportunity and do our best to replace outdated terms with their more inclusive alternatives. As you can see in [1] the TripleO project is already working on this initiative, and I would like to work on this as well on the tempest side. Any thoughts? Shall I start with a sepc, adding deprecation warnings? [1] https://review.opendev.org/#/c/740013/1/specs/victoria/renaming_rules.rst Kind regards, -- Arx Cruz Software Engineer Red Hat EMEA <https://www.redhat.com> arxcruz(a)redhat.com @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

6 9

[infra][magnum][ci] Issues installing bashate and coverage
by Spyros Trigazis 05 Jan '21

05 Jan '21

Hello infra, openstack-tox-lower-constraints fails for bashate and coverage. (Maybe more, I bumped bashate and it failed for coverage. I don;t want to waste more resources on our CI) eg https://review.opendev.org/c/openstack/magnum/+/765881 https://review.opendev.org/c/openstack/magnum/+/765979 Do we miss something? Thanks, Spyros

4 6

[Ussuri] Auto shutdown VM
by Deepa KR 04 Jan '21

04 Jan '21

Hi All We have a Openstack setup with the Ussuri Version and I am regularly facing auto shutdown of a few VMs (ubuntu16.04) randomly . If I restart then the instance is back . From logs I was able to see the messages below . WARNING nova.compute.manager [req-2a21d455-ac04-44aa-b248-4776e5109013 813f3fb52c434e38991bb90aa4771541 10b5279cb6f64ca19871f132a2cee1a3 - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Received unexpected event network-vif-unplugged-e97839a1-bbc4-4d26-af30-768ca3630ce9 for instance with vm_state active and task_state None. INFO nova.compute.manager [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] VM Stopped (Lifecycle Event) INFO nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. syslog:Nov 13 07:01:07 fgshwbucehyp04 nova-compute[2680204]: 2020-11-13 07:01:07.684 2680204 WARNING nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4 nova.compute.manager [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance is already powered off in the hypervisor when stop is called. nova.virt.libvirt.driver [req-8261f607-4f1e-459d-85d4-e269694dd477 - - - - -] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance already shutdown. nova.virt.libvirt.driver [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully. nova.compute.manager [req-7a0a0d03-e286-42f0-9e36-38a432f236f3 d9ca03b9d0884d51a26a39b6c82f02eb 304d859c43df4de4944ca5623f7f455c - default default] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Get console output nova.virt.libvirt.driver [-] [instance: 28cd861c-ef15-444a-a902-9cac643c72b5] Instance destroyed successfully. I searched a few blogs and forums but couldn't find a solution to it . Few mentioned to add sync_power_state_interval=-1 in /etc/nova/nova.conf .But understood that this will help only when nova stops vm. But in this case vm itself is shutting down (Instance shutdown by itself. Calling the stop API) Also no memory issue in VM nor the hypervisor. Also did apt-get upgrade . It would be great if anyone can shed light to this issue. Regards, Deepa K R Sent from my iPhone

7 15

[SIG] [First Contact] First Meeting of the Year!
by Kendall Nelson 04 Jan '21

04 Jan '21

Hello! We will be holding our meeting tomorrow (at 00:00 UTC in #openstack-upstream-institute) with a pretty short agenda[1] so feel free to add as you see fit! One thing I do want to try is to set some concrete goal or two for this year to work towards. So if you're new to the SIG this is the perfect time to come meet us and get involved! -Kendall (diablo_rojo) [1] https://wiki.openstack.org/wiki/First_Contact_SIG#Meeting_Agenda

1 0

[tc] weekly meeting
by Mohammed Naser 04 Jan '21

04 Jan '21

Hi there, Hope everyone enjoyed the holidays! Here's an update on what happened in the OpenStack TC these past few days. You can get more information by checking for changes in openstack/governance repository. # Patches ## Open Reviews - Add Resolution of TC stance on the OpenStackClient | https://review.opendev.org/c/openstack/governance/+/759904 - Remove Karbor project team | https://review.opendev.org/c/openstack/governance/+/767056 - Add glance-tempest-plugin to Glance | https://review.opendev.org/c/openstack/governance/+/767666 ## Other Reminders - Our next TC Weekly meeting is scheduled for January 7th at 1500 UTC. If you would like to add topics for discussion, please go to https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting and fill out your suggestions by Wednesday, January 6th, at 2100 UTC. Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 0

[tripleo] next irc meeting Tuesday Jan 05 @ 1400 UTC in #tripleo
by Marios Andreou 04 Jan '21

04 Jan '21

The next TripleO irc meeting is: ** Tuesday 05th January at 1400 UTC in #tripleo. ** ** https://wiki.openstack.org/wiki/Meetings/TripleO ** ** https://etherpad.opendev.org/p/tripleo-meeting-items ** Please add any one-off items you want to hilight at https://etherpad.opendev.org/p/tripleo-meeting-items This can include review requests, blocking issues or to socialise ongoing or planned work, or anything else you want to share. Our last meeting was held on Dec 22nd - you can find the logs there http://eavesdrop.openstack.org/meetings/tripleo/2020/tripleo.2020-12-22-14.… Hope to see you there!

1 0

[swift] Openstack controller/keystone cause not uploading file to swift storage
by Hamidreza Hosseini 04 Jan '21

04 Jan '21

Hi, I've installed swift storage and swift-proxy and keystone for object storage I'm using memcached for caching keystone's data But suddenly in some moments my cpu of keystone servers goes up till 50% (it is apache and keystone proccess) and the swift proxies wouldn't upload anything!!! As I said servers have free cpu and ram and kernel tuned well which parameter should I check for my problem? How can I solve this strange problem?

1 0

some error abount ironic-python-agent-builder
by Ankele zhang 04 Jan '21

04 Jan '21

Hi~ I have an OpenStack platform in Rocky version. I use ironic-python-agent-builder to build a tinyipa image to customing HardwareManager for 'RAID configuration' in Ironic cleaning steps. While I follow the steps ' https://docs.openstack.org/ironic-python-agent-builder/latest/admin/tinyipa…' to build tinyipa image, it occurs error as following: [image: image.png] [image: image.png] so, what the IPA_SOURCE_DIR means? Do I need to download the source code of the ironic-python-agent and copy it to /opt/stack/ before this? Looking forward to your reply. Ankele

2 1

[neutron] Bug deputy report December 28th to January 3rd
by Lajos Katona 04 Jan '21

04 Jan '21

Hi, I was Neutron bug deputy for this week, and it was a really quiet week. Opinion: - https://bugs.launchpad.net/neutron/+bug/1909160 : high cpu usage when listing security groups - The issues is reported from Rocky - On openstack-discuss was some mail about slow security-group API on Stein for example: http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019573… - I checked some rally execution from latest master runs and haven't seen such Happy New Year! Lajos Katona (lajoskatona)

1 0

[neutron] CI meeting - Tuesday 05.01.2021
by Slawek Kaplonski 04 Jan '21

04 Jan '21

Hi, I just found out that I have some internal training tomorrow and will not be able to chair our tomorrow's CI meeting. As this is just second day after holidays season for many of us, lets cancel this one more meeting. If there is anything urgent related to the CI, please ping me on IRC or send me an email about it. See You all on the meeting next week. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[neutron] Team meeting - Tuesday 05.01.2021
by Slawek Kaplonski 04 Jan '21

04 Jan '21

Hi, I just found out that I have some internal training tomorrow and will not be able to chair our tomorrow's team meeting. As this is just second day after holidays season for many of us, lets cancel this one more meeting. See You all on the meeting next week. Happy New Year to all of You :) -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

failed to launch openstack instance when integrate with odl
by john smith 02 Jan '21

02 Jan '21

hello I tried to integrate Openstack train with Opendaylight magnesium and my references were [1] <https://docs.opendaylight.org/projects/netvirt/en/stable-magnesium/openstac…> and [2] <https://files.openstack.org/docs/networking-odl/train/install/installation.…> but after doing all the steps, the instance failed to launch. I changed port_binding_controller in [ml2_odl] section in the ml2_conf.ini file from pseudo-agentdb-binding to legacy-port-binding and then the instance launched but the status of router interfaces was still down. I have a controller node and a compute node. and Opendaylight runs on the controller node. nova-compute.log: INFO nova.virt.libvirt.driver [-] [instance: 975fa79e-6567-4385-be87-9d12a8eb3e94] Instance destroyed successfully. 2021-01-02 12:33:23.383 25919 ERROR vif_plug_ovs.ovsdb.impl_vsctl [req-a0a7ebf0-7e63-4c60-a8d2-07c05f1aa4f4 04c7685a2166481a9ace54eb5e71f6e5 ca28ee1038254649ad133d5f09f7a186 - default default] Unable to execute ['ovs-vsctl', '--timeout=120', '--oneline', '--format=json', '--db=tcp: 127.0.0.1:6640', '--', '--if-exists', 'del-port', 'br-int', 'tap50eb0b68-a4']. Exception: Unexpected error while running command. Command: ovs-vsctl --timeout=120 --oneline --format=json --db=tcp: 127.0.0.1:6640 -- --if-exists del-port br-int tap50eb0b68-a4 Exit code: 1 Stdout: '' Stderr: 'ovs-vsctl: tcp:127.0.0.1:6640: database connection failed (Connection refused)\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. controller ovs: Manager "tcp:192.168.222.48:6640" is_connected: true Bridge br-int Controller "tcp:192.168.222.48:6653" is_connected: true fail_mode: secure Port tune52c5c73a50 Interface tune52c5c73a50 type: vxlan options: {key=flow, local_ip="10.0.0.31", remote_ip="10.0.0.11"} Port br-int Interface br-int type: internal Bridge br-ex Port br-ex Interface br-ex type: internal Port ens160 Interface ens160 ovs_version: "2.13.1" compute ovs: Manager "tcp:192.168.222.48:6640" is_connected: true Bridge br-int Controller "tcp:192.168.222.48:6653" is_connected: true fail_mode: secure Port br-int Interface br-int type: internal Port tun34b3712d975 Interface tun34b3712d975 type: vxlan options: {key=flow, local_ip="10.0.0.11", remote_ip="10.0.0.11"} Port tund5123ce5b8a Interface tund5123ce5b8a type: vxlan options: {key=flow, local_ip="10.0.0.11", remote_ip="10.0.0.31"} ovs_version: "2.13.1"

1 0

[puppet][neutron] Status of networking-cisco and networking-bigswitch
by Takashi Kajinami 02 Jan '21

02 Jan '21

Hello, While cleaning up outdated implementations in puppet-neutron, I noticed that the following 2 plugins are supported by puppet-neutron but the plugins are no longer maintained actively these days. 1) networking-cisco https://opendev.org/x/networking-cisco This repo has not got any updates for these 2 year except fo opendev migration and supports not Python 3.6 but only Python 2.7 and Python 3.5. 2) networking-bigswitch https://opendev.org/x/networking-bigswitch This repo was updated 10 months ago, but doesn't have ussuri and victoria branch created. If I don't hear any plan to keep maintaining these 2 repositories I'll deprecate support for these 2 plugins in this release and remove the support completely in the next cycle. Thank you, Takashi Kajinami

1 0

[Ceilometer][all]Visualising Ceilometer
by Adekunbi Adewojo 31 Dec '20

31 Dec '20

Hi there, I want to get the results in ceilometer into Grafana dashboard so that I can visualise them. Is there a manual on this or can someone point me in the right direction please? Currently, I interact with ceilometer using it's API from my code to get some metrics result but would love to visualise them. Also, I came across Sensu and thinks this could help. Can it be used to visualise metrics gotten from ceilometer? I am interested in getting VM metrics in my code and at the same time be able to visualise them when applications on the VM are stress tested. Thanks.

3 3

[tacker] Next meeting Tuesday 12 Jan
by Yasufumi Ogawa 31 Dec '20

31 Dec '20

Hi team, As we agreed in the last meeting, we will skip IRC meeting next week for new year holidays. Thank you & happy holidays! Yasufumi

1 0

[horizon] No meetings until January, 13th
by Ivan Kolodyazhny 30 Dec '20

30 Dec '20

Hi team, As agreed during the last IRC meeting [1], we'll skip the next two meetings due to the Christmas and New Year holidays. [1] http://eavesdrop.openstack.org/meetings/horizon/2020/horizon.2020-12-23-15.… Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 0

Openstack instance management prive
by Ranjeet Kumar 30 Dec '20

30 Dec '20

Hi What could be the average unit price for the life cycle management of an instance in open stack environment?

1 0

Instance on provider network not working
by mohammad mighani 29 Dec '20

29 Dec '20

Hello I use openstack Ussuri in Ubuntu 18 and one controller node and one compute node with two interface, management and Provider that connected to internet i install keystone, glance, placement, nova, neutron on self service option *what i can * i can create provider network and self service network and router between them i can ping router i can launch instance on self service network and it has access to internet and everything looks fine *what i can NOT* I can't launch instances on the provider network. When I launch an instance on the provider network, the provider interface of the compute node will be corrupted and not respond and the instance will not be created. error in instance : *Message *Build of instance 2a86a84b-50fc-415c-a15c-cb87398f37ea aborted: Failed to allocate the network(s), not rescheduling. *Code *500 *Details *Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6529, in _create_domain_and_network post_xml_callback=post_xml_callback) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 513, in wait_for_instance_event actual_event = event.wait() File "/usr/lib/python3/dist-packages/eventlet/event.py", line 125, in wait result = hub.switch() File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 298, in switch return self.greenlet.switch() eventlet.timeout.Timeout: 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2378, in _build_and_run_instance accel_info=accel_info) File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3663, in spawn cleanup_instance_disks=created_disks) File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6552, in _create_domain_and_network raise exception.VirtualInterfaceCreateException() nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2200, in _do_build_and_run_instance filter_properties, request_spec, accel_uuids) File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2444, in _build_and_run_instance reason=msg) nova.exception.BuildAbortException: Build of instance 2a86a84b-50fc-415c-a15c-cb87398f37ea aborted: Failed to allocate the network(s), not rescheduling. *Created *Dec. 29, 2020, 9:18 a.m. *After this I can't ping the computer from the provider network interface.* I reinstalled openstack so many times and also I tried the train and victoria version and have the same problem. I attached log files of nova compute and neutron linux bridge from compute node and neutron server from controller node. thanks.

2 1

[neutron] Team meetings this year cancelled
by Slawek Kaplonski 29 Dec '20

29 Dec '20

Hi, As we discussed during our last meeting, we are going to cancel our team meetings in next 2 weeks. Have a great holidays and see You all on the meeting at 5.01.2021 -- Slawek Kaplonski Principal Software Engineer Red Hat

2 1

[Nova][queens] disk.config not found issue
by Ignazio Cassano 29 Dec '20

29 Dec '20

Hello All, after a compute node failure if I try to reset state and hard reboot an instance it fails because disk.config file does not exist. I am using cinder with netapp nfs driver and I am using disk config drive. It happens also on stein. Any help, please ? It can be' solved creating the disk.config with touch command but it does not seem a good solution . Ignazio

1 1

[osc][neutron] security group list slow in Stein
by Eric K. Miller 28 Dec '20

28 Dec '20

Hi, We are working on upgrade planning for our Stein deployments, but since we have some time before this will be finalized, I thought I'd ask if there has been any performance improvement/fixes in the performance of the "security group list" command in the OpenStack Client and/or Neutron API. "security group rule list" is nearly instant (when authenticated to a project), and returns more info than security group list, but "security group list" can take up to 29 seconds in the worst case when authenticated to a project. Oddly enough, when logged in as cloud admin, "security group list" only takes about 14 seconds, even though it is displaying every security group in the cloud (in our case, quite a few!). Also, just wondering if anyone else has this issue. Thanks! Eric

1 0

[neutron] bug deputy report December 21st to 27th
by Miguel Lavalle 28 Dec '20

28 Dec '20

Hi, This week Slawek and I cooperated to perform the bug deputy duties. It was a relatively quiet week. These are the bugs triaged during the week: High ==== https://bugs.launchpad.net/neutron/+bug/1908957 iptable rules collision deployed with k8s iptables kube-proxy enabled. Owner: Norman Shen https://bugs.launchpad.net/neutron/+bug/1909038 [ovn] TypeError: lrp_set_options() takes 2 positional arguments but 3 were given. Owner: Flavio Fernandes https://bugs.launchpad.net/neutron/+bug/1909234 [fullstack] "test_min_bw_qos_port_removed" failing randomly. Owner: Rodolfo Alonso RFE === https://bugs.launchpad.net/neutron/+bug/1909100 [RFE]add new vnic type "cyborg"

1 0

Re: something abount ironic-python-agent
by Kaifeng Wang 28 Dec '20

28 Dec '20

On Sun, Dec 27, 2020 at 8:39 PM Ankele zhang <ankelezhang(a)gmail.com> wrote: > Hi Kaifeng: > Before receiving your reply, I have downloaded > the ronic-python-agent-builder-2.3.0, but I don't know how to build it and > how to use it, I can't find the direction for use. > And I try 'python setup.py install' to install it. and the following is my > execution. > > I don't know the optional values of 'distribution', 'release', 'element', > I hope you can help me. > > distribution refers to the base image that the ramdisk will be built on, release is used to specify the distribution variant. element is a concept of diskimage-builder, each element is handling part of the building process, you can take a look at the diskimage-builder documentation for a complete list [1]. In case further customization on the ramdisk is required, elements can be included by the argument. We have some examples on how to use the IPA-B at [2]. [1] https://docs.openstack.org/diskimage-builder/latest/elements.html [2] https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Ankele > > Kaifeng Wang <kaifeng.w(a)gmail.com> 于2020年12月27日周日下午2:04写道： > >> >> >> On Sun, Dec 27, 2020 at 12:12 AM Ankele zhang <ankelezhang(a)gmail.com> >> wrote: >> >>> Hi ~ >>> I have deployed OpenStack platform in Rocky version and integreted >>> Ironic(Rocky) into it according to the official documents. I download the >>> coreOS vmlinuz and ramdisk images from >>> https://tarballs.opendev.org/openstack/ironic-python-agent/coreos/files/. >>> and if I don't clean my devices I can deploy baremetal node server >>> currently. When I cleaning my disk using "openstack baremetal node clean >>> --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]' my_bm", >>> I got an error message "Clean step failed: Error performing clean_step >>> erase_devices: No HardwareManager found to handle method: Could not find >>> method: erase_block_device" from ironic-conductor.log and I got the same >>> error in coreOS by "sudo journalctl -u ironic-python-agent -f". I download >>> the Rocky ironic-python-agent source code from github and I search >>> 'erase_block_devices' ,it returned from hardware.py +781. So, the first >>> problem , I don't know why I got this error message. >>> >> >> CoreOS based ramdisk was not maintained for a while, you could try a >> rocky release at >> https://tarballs.opendev.org/openstack/ironic-python-agent/dib/ >> >> >>> I downloaded the source code of the ironic-python-agent and tried to >>> build the coreOS image by 'Makefile', but I always got error as following, >>> this is my second problem. >>> >>> >> Building an IPA ramdisk is moved from diskimage-builder to >> ironic-python-agent-builder, you can find the document here: >> https://docs.openstack.org/ironic-python-agent-builder/latest/ >> >> >>> >>> Finally, I want to code my own 'custom HardwareManager' to support 'raid >>> config' for the ironic-python-agent, but I don't know how to get started. >>> >>> >> For an in-band RAID support, you'll need to create a new hardware manager >> and implement some deploy steps to support RAID configuration, >> specifically, the create_configuration and delete_configuration steps. >> The following links may provide some starting points for you. >> https://docs.openstack.org/ironic/latest/contributor/deploy-steps.html >> >> https://docs.openstack.org/ironic/latest/admin/cleaning.html#raid-interface >> >> Hope this helps. >> >> >>> Looking forward to your help. Thank you. >>> >>> Ankele >>> >>

1 0

something abount ironic-python-agent
by Ankele zhang 26 Dec '20

26 Dec '20

Hi ~ I have deployed OpenStack platform in Rocky version and integreted Ironic(Rocky) into it according to the official documents. I download the coreOS vmlinuz and ramdisk images from https://tarballs.opendev.org/openstack/ironic-python-agent/coreos/files/. and if I don't clean my devices I can deploy baremetal node server currently. When I cleaning my disk using "openstack baremetal node clean --clean-steps '[{"interface": "deploy", "step": "erase_devices"}]' my_bm", I got an error message "Clean step failed: Error performing clean_step erase_devices: No HardwareManager found to handle method: Could not find method: erase_block_device" from ironic-conductor.log and I got the same error in coreOS by "sudo journalctl -u ironic-python-agent -f". I download the Rocky ironic-python-agent source code from github and I search 'erase_block_devices' ,it returned from hardware.py +781. So, the first problem , I don't know why I got this error message. I downloaded the source code of the ironic-python-agent and tried to build the coreOS image by 'Makefile', but I always got error as following, this is my second problem. [image: image.png] Finally, I want to code my own 'custom HardwareManager' to support 'raid config' for the ironic-python-agent, but I don't know how to get started. Looking forward to your help. Thank you. Ankele

2 1

ssh authentication error with Gerrit
by Pete Zaitcev 26 Dec '20

26 Dec '20

Hello: Does anyone here happen to know how to deal with something like this: ........... debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug1: Server host key: ssh-rsa SHA256:RXNl/GKyDaKiIQ93BoDvrNSKUPFvA1PNeAO9QiirYZU debug1: Host '[review.opendev.org]:29418' is known and matches the RSA host key. debug1: Found key in /q/zaitcev/.ssh/known_hosts:133 debug1: rekey out after 4294967296 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: rekey in after 4294967296 blocks debug1: Will attempt key: /q/zaitcev/.ssh/id_rsa_ostk2014 RSA SHA256:nz5*** explicit agent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey debug1: Next authentication method: publickey debug1: Offering public key: /q/zaitcev/.ssh/id_rsa_ostk2014 RSA SHA256:nz5*** explicit agent debug1: send_pubkey_test: no mutual signature algorithm debug1: No more authentication methods to try. zaitcev(a)review.opendev.org: Permission denied (publickey). [zaitcev@suzdal swift-dark]$ I guess that ssh client in Fedora 33 has a cipher suite that has no intersection with the ssh server at review.opendev.org. But I do not understand what the server is offering, so I do not know what I need to enable. Thanks in advance, -- Pete

2 2

[qa][tempest-plugins][release][stable] Releasing stein-em tag for Tempest plugins
by Ghanshyam Mann 26 Dec '20

26 Dec '20

Hello Everyone, As the stable/stein branch is in Extended Maintenance now[1], Tempest has dropped the support of stable/stein[2]. Tempest and its plugins are branchless which means the master version of Tempest and its plugins are used to test the supported stable branches. Once the stable branch is moved to EM state then, Tempest and its plugins compatible tag needs to be released so that we can keep testing the EM stable branches with this tag once the master Tempest and its plugins are not compatible[3]. This tag can be used as the latest compatible version of Tempest and its plugins for testing the stable/stein in upstream as well as production cloud testing. I have proposed all release patches: - https://review.opendev.org/q/topic:%22tempest-plugin-stein-em%22+(status:op… [1] https://releases.openstack.org/stein/index.html [2] https://review.opendev.org/c/openstack/tempest/+/766770 [3] https://docs.openstack.org/tempest/latest/stable_branch_support_policy.html -gmann

1 0

grouping resources on specific set of nodes
by Volodymyr Litovka 26 Dec '20

26 Dec '20

Hi colleagues, I can't realize how to do the simple-looking thing: there are few locations with small amount of compute/storage/network resources (like small branches with 3 physical servers for both compute, storage and network roles) and I want to see all these locations as single cloud and manage them centrally using single "controller cluster", while localizing resource usage to the specific location. _Kind of_ assign specific project to location "A" and, thus, all resources (VMs, networks, storage resources, ...) in this project will be allocated from nodes, physically located in location "A". Are there any ways to implement this kind of behavior in Openstack? Thank you. -- Volodymyr Litovka "Vision without Execution is Hallucination." -- Thomas Edison

1 0

New nova compute nodes not correctly adding into service
by Grant Morley 25 Dec '20

25 Dec '20

Hi All, I was wondering if anyone could point me in the right direction with an issue we are having adding in new compute nodes to our platform. We have been trying to add a few new compute nodes to increase our capacity and upon configuring them with OSA and them appearing to be in service we have noticed that no new instances are provisioning onto them. We did a test by disabling all compute nodes except for the new ones and an instance launch failed with "No valid compute hosts were found" We have enabled debug mode on Nova and also looked at the nova-api containers ( scheduler, compute etc ) and it appears there is not even an attempt to try and launch an instance on the new hosts from what we can see. The only thing we have noticed on the newly added hosts is that the "privsep-helper" doesn't appear to be started whereas on a working node it is running. We have not changed the software versions of OpenStack or OSA, we are running Queens. The newly added compute node is reporting itself back fine from what we can see: Total usable vcpus: 72, total allocated vcpus: 0 _report_final_resource_view /openstack/venvs/nova-17.1.2/lib/python2.7/site-packages/nova/compute/resource_tracker.py:827 2020-12-24 16:26:32.895 6491 INFO nova.compute.resource_tracker [req-6c58a97e-b7cb-4987-a219-da5b1ce156f8 - - - - -] Final resource view: name=compute-27.openstack.local phys_ram=257826MB used_ram=2048MB phys_disk=30166GB used_disk=0GB total_vcpus=72 used_vcpus=0 pci_stats=[] 2020-12-24 16:26:33.022 6491 DEBUG nova.compute.resource_tracker [req-6c58a97e-b7cb-4987-a219-da5b1ce156f8 - - - - -] Compute_service record updated for compute-27:compute-27.openstack.local _update_available_resource /openstack/venvs/nova-17.1.2/lib/python2.7/site-packages/nova/compute/resource_tracker.py:767 2020-12-24 16:26:33.023 6491 DEBUG oslo_concurrency.lockutils [req-6c58a97e-b7cb-4987-a219-da5b1ce156f8 - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker._update_available_resource" :: held 0.334s inner /openstack/venvs/nova-17.1.2/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285 2020-12-24 16:26:35.001 6491 DEBUG oslo_service.periodic_task [req-6c58a97e-b7cb-4987-a219-da5b1ce156f8 - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks /openstack/venvs/nova-17.1.2/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 Does anyone know of any other logs we can check / enable that might lead us in the right direction to figure out why the new hosts are not adding in? Any help would be greatly appreciated as we are a bit stumped at the moment. Many thanks, -- Grant Morley Site Reliability Engineer, Civo Ltd Unit H-K, Gateway 1000, Whittle Way Stevenage, Herts, SG1 2FP, UK Visit us at www.civo.com <https://www.civo.com/> Signup for an account now > <https://www.civo.com/signup>

3 2

[cinder] patches for lower-constraints
by Brian Rosmaita 23 Dec '20

23 Dec '20

As discussed at today's cinder meeting, there are patches available that address the lower constraints problems arising from the upgraded pip dependency resolver for most cinder deliverables. Except for master, I've tried to keep the changes minimal. One possibly controversial change has been to add indirect dependencies to test-requirements.txt in a few cases where the version range was so wide that the resolver was taking a very long time to figure out a satisfactory resolution. An example of this is the cinder stable/train patch: https://review.opendev.org/c/openstack/cinder/+/767954 I'm not sure when these patches will be merged, but you can always add the appropriate one as a dependency in gerrit to get the check jobs to pass on your patch. You can find them by using the gerrit hashtag 'fix-l-c': https://review.opendev.org/q/hashtag:fix-l-c Apparently only the owner of a review can add a hashtag, so os-brick/master (merged, owned by lpetrut) and the patches for the non-releaseable stable branches that e0ne is working on aren't in the list yet.

1 0

[cinder] limited review bandwidth until 4 January 2021
by Brian Rosmaita 23 Dec '20

23 Dec '20

Just a reminder that most cinder cores will not be active from tomorrow (24 December) through 4 January 2021. Happy holidays! brian

1 0

[cinder] meeting time change poll
by Brian Rosmaita 23 Dec '20

23 Dec '20

At today's meeting, Lucio proposed moving the cinder weekly meeting time. Please respond to this poll to assess some options before Tuesday 22 December at 1200 UTC. https://rosmaita.wufoo.com/forms/wallaby-cinder-meeting-time-poll/ There's a free-form field on the form so you can propose other alternatives. We aren't considering changing the day of the meeting (Wednesday) at this time, but that's a possibility if it would encourage more attendance. NOTE: next week's meeting, Wednesday 23 December, will be held at the usual time of 1400 UTC. cheers, brian

1 1

[tc] weekly meeting summary
by Mohammed Naser 23 Dec '20

23 Dec '20

Hi everyone, Here's a summary of what happened in our TC weekly meeting last Thursday, Dec 17th. # ATTENDEES (LINES SAID) 1. mnaser (82) 2. gmann (46) 3. diablo_rojo (25) 4. jungleboyj (18) 5. fungi (10) 6. belmoreira (6) 7. ricolin (4) # MEETING SUMMARY 1. Rollcall 2. Skipping the next meetings. Due to the upcoming holidays, we are skipping the next two meetings on Dec 24 and Dec 31st. Mnaser to send an email with details. 3. Follow up on past action items - mnaser send email to ML to find volunteers to help drive goal selection: email sent, and no one strongly opposed. mnaser will write a proposed goal for X about stabilization/cooldown. - gmann complete retirement of searchlight & qinling: DONE - diablo_rojo complete retirement of karbor: In progress, getting there. 4. Audit SIG list and chairs (diablo_rojo) - This list is out of date and a lot of chairs are not active anymore. If any SIG has completed their purpose then we move them to an advisory state, otherwise close. diablo_rojo reach out to SIGs/ML and start auditing states of SIGs 5. Annual report suggestions (diablo_rojo) - mnaser and diablo_rojo are working on the annual report for OpenStack, ideas are welcome. Once it's mostly written a draft will be sent out before submitting it. 6. X cycle goal selection start This item has to go with the action item above regarding the stabilization goal proposal. We'll remove this item from the agenda. 7. Audit and clean-up tags (gmann) - http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019505… gmann started ML on API tag and will see how many projects start and will continue on other tag audits in parallel. We'll remove sub-items in there in the meantime. gmann continue to audit tags + outreach to the community to apply for them 8. X cycle release name vote recording (gmann) - Votes are recorded in ML, mnaser will remove X cycle release name vote recording from the agenda. 9. CentOS 8 releases are discontinued / switch to CentOS 8 Stream (gmann/yoctozepto) - We have an action item to get the community to get together and the QA team is doing the right thing. We'll remove centos 8 topic from the upcoming agenda. 10. Open reviews - https://review.opendev.org/q/projects:openstack/governance+is:open - https://review.opendev.org/c/openstack/governance/+/759904 - https://review.opendev.org/c/openstack/governance/+/759904/7/resolutions/20… # ACTION ITEMS - mnaser send out email about skipping both upcoming meetings - mnaser write a proposed goal for X about stabilization/cooldown - diablo_rojo complete retirement of karbor - diablo_rojo reach out to SIGs/ML and start auditing states of SIGs - mnaser drop X cycle goal selection start from agenda - gmann continue to audit tags + outreach to the community to apply for them - mnaser drop X cycle release name vote recording - mnaser remove centos 8 topic from upcoming agenda To read the full logs of the meeting, please refer to http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-12-17-15.00.log.html -- Mohammed Naser VEXXHOST, Inc.

1 0

[tc] weekly meeting skip
by Mohammed Naser 23 Dec '20

23 Dec '20

Hi there, Due to the upcoming holidays, we are skipping the next TC meetings on Dec 24th and Dec 31st. We will resume with our regular schedule on Thursday, January 7th. Thank you & happy holidays! Regards Mohammed -- Mohammed Naser VEXXHOST, Inc.

1 0

oslo message direct_mandatory_flag question
by Satish Patel 22 Dec '20

22 Dec '20

Folks, I am getting a very strange error from time to time on senlin logs and when it hits this error i can't do anything in senlin until i restart service. After googling i found this bug https://bugs.launchpad.net/oslo.messaging/+bug/1905965 It's pretty much related to my issue because I am running the latest victoria. Dec 22 13:56:44 os-lab-infra-1-senlin-container-16f24bbe senlin-wsgi-api[8188]: 2020-12-22 13:56:44.212 8188 ERROR oslo.messaging._drivers.impl_rabbit [-] [df314561-6415-4103-a1fd-14ab95182cfb] AMQP server on 10.65.6.176:5671 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error> Dec 22 13:56:44 os-lab-infra-1-senlin-container-16f24bbe senlin-wsgi-api[8188]: 2020-12-22 13:56:44.220 8188 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Dec 22 13:56:44 os-lab-infra-1-senlin-container-16f24bbe senlin-conductor[8250]: 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server [req-3c89475b-89fc-404b-8537-df7a587261d9 462618bed32745d2a9166bcc33fc117e f1502c79c70f4651be8ffc7b844b584f - - -] MessageUndeliverable error, source exception: Basic.return: (312) NO_ROUTE, routing_key: reply_54d93c43fe894ed18ce8092f4497306b, exchange: : : oslo_messaging.exceptions.MessageUndeliverable 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 184, in _process_incoming 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server message.reply(res) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 149, in reply 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self._send_reply(conn, reply, failure) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 125, in _send_reply 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg)) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1320, in direct_send 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self._ensure_publishing(self._publish_and_raises_on_missing_exchange, 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1202, in _ensure_publishing 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self.ensure(method, retry=retry, error_callback=_error_callback) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 826, in ensure 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server ret, channel = autoretry_method() 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/kombu/connection.py", line 525, in _ensured 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server return fun(*args, **kwargs) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/kombu/connection.py", line 601, in __call__ 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server return fun(*args, channel=channels[0], **kwargs), channels[0] 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 815, in execute_method 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server method() 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1294, in _publish_and_raises_on_missing_exchange 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self._publish(exchange, msg, routing_key=routing_key, 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1238, in _publish 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self._producer.publish( 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/kombu/messaging.py", line 175, in publish 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server return _publish( 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/kombu/messaging.py", line 197, in _publish 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server return channel.basic_publish( 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/channel.py", line 1782, in basic_publish_confirm 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self.wait([spec.Basic.Ack, spec.Basic.Nack], callback=confirm_handler) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/abstract_channel.py", line 86, in wait 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server self.connection.drain_events(timeout=timeout) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/connection.py", line 514, in drain_events 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server while not self.blocking_read(timeout): 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/connection.py", line 520, in blocking_read 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server return self.on_inbound_frame(frame) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/method_framing.py", line 77, in on_frame 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server callback(channel, msg.frame_method, msg.frame_args, msg) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/connection.py", line 526, in on_inbound_method 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server return self.channels[channel_id].dispatch_method( 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/abstract_channel.py", line 143, in dispatch_method 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server listener(*args) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/amqp/channel.py", line 2006, in _on_basic_return 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server callback(exc, exchange, routing_key, message) 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server File "/openstack/venvs/senlin-22.0.0.0b2.dev56/lib/python3.8/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 852, in on_return 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server raise exceptions.MessageUndeliverable(exception, exchange, routing_key, 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server oslo_messaging.exceptions.MessageUndeliverable 2020-12-22 13:56:44.461 8250 ERROR oslo_messaging.rpc.server Dec 22 13:56:45 os-lab-infra-1-senlin-container-16f24bbe senlin-wsgi-api[8188]: 2020-12-22 13:56:45.248 8188 INFO oslo.messaging._drivers.impl_rabbit [-] [df314561-6415-4103-a1fd-14ab95182cfb] Reconnected to AMQP server on 10.65.6.176:5671 via [amqp] client with port 44246. Dec 22 13:56:53 os-lab-infra-1-senlin-container-16f24bbe senlin-health-manager[8218]: 2020-12-22 13:56:53.954 8218 INFO senlin.engine.health_manager [req-c0118d24-9f32-47c8-926e-eb4b60f922cd 462618bed32745d2a9166bcc33fc117e f1502c79c70f4651be8ffc7b844b584f - - -] Health check passed for all nodes in cluster f51b3154-fb8e-43f2-b68b-03b091c1e0bf. Dec 22 13:57:23 os-lab-infra-1-senlin-container-16f24bbe senlin-health-manager[8218]: 2020-12-22 13:57:23.962 8218 INFO senlin.engine.health_manager [req-472f4403-fd66-422a-8a84-a0b0f51d94cc 462618bed32745d2a9166bcc33fc117e f1502c79c70f4651be8ffc7b844b584f - - -] Health check passed for all nodes in cluster f51b3154-fb8e-43f2-b68b-03b091c1e0bf. Dec 22 13:57:23 os-lab-infra-1-senlin-container-16f24bbe senlin-health-manager[8218]: 2020-12-22 13:57:23.963 8218 INFO senlin.engine.health_manager [req-a5ff1fe3-35e8-443b-b91a-aa1d78656569 462618bed32745d2a9166bcc33fc117e f1502c79c70f4651be8ffc7b844b584f - - -] Health check passed for all nodes in cluster f51b3154-fb8e-43f2-b68b-03b091c1e0bf. Dec 22 13:57:44 os-lab-infra-1-senlin-container-16f24bbe uwsgi[8188]: Tue Dec 22 13:57:44 2020 - uwsgi_response_writev_headers_and_body_do(): Connection reset by peer [core/writer.c line 306] during GET /v1/clusters?global_project=False (10.65.6.17) Dec 22 13:57:44 os-lab-infra-1-senlin-container-16f24bbe senlin-wsgi-api[8188]: 2020-12-22 13:57:44.448 8188 CRITICAL senlin-api [req-3c89475b-89fc-404b-8537-df7a587261d9 462618bed32745d2a9166bcc33fc117e f1502c79c70f4651be8ffc7b844b584f - - -] Unhandled error: OSError: write error 2020-12-22 13:57:44.448 8188 ERROR senlin-api OSError: write error 2020-12-22 13:57:44.448 8188 ERROR senlin-api Dec 22 13:57:53 os-lab-infra-1-senlin-container-16f24bbe senlin-health-manager[8218]: 2020-12-22 13:57:53.955 8218 INFO senlin.engine.health_manager [req-8a578e8e-438d-45c1-b81d-c6f3aed5fc81 462618bed32745d2a9166bcc33fc117e f1502c79c70f4651be8ffc7b844b584f - - -] Health check passed for all nodes in cluster f51b3154-fb8e-43f2-b68b-03b091c1e0bf. As per advice i have tired to add following snippet to fix this issue [DEFAULT] transport_url = rabbit://senlin:94d7aecb853145779db8f1dcb@10.65.6.176:5671//senlin?ssl=1 [oslo_messaging_rabbit] ssl = True direct_mandatory_flag = False But i got error when trying to set direct_mandatory_flag option ERROR oslo_service.service [-] Error starting t hread.: oslo_config.cfg.ConfigFileValueError: Value for option direct_mandatory_flag from LocationInfo(location=<Locations.user: (4, True)>, detail='/etc/senlin/senlin.conf') is not valid: invalid literal for int() with base 10: 'False' Any idea what is going on here?

2 2

[infra][tripleo] is it possible to control (tripleo) gate queue priority?
by Marios Andreou 22 Dec '20

22 Dec '20

Hello illustrious members of openstack infra tripleo-ci squad is wondering if it is possible for (some subset of) us to be able to set the priority of a particular patch/es in the tripleo queue. We've done this "manually" in the past, by abandoning all patches in the gate & then restoring in order and putting the priority patch at the top of the dependency queue. However abandoning all the things is completely disruptive for everyone else (sometimes that might be necessary if your queue is way too long but still...). So the question is, is there a better way to put a particular patch at the top of our queue when we need to do that? thanks for your thoughts, sorry if this has come up before I couldn't quickly find something in the list archives. regards, marios

4 5

[off-topic][tripleo] Merry Christmas!
by Carlos Camacho Gonzalez 22 Dec '20

22 Dec '20

Hi everyone! Its been a complex year for everyone, however, we managed to do amazing things, I hope we can get a boost of energy these holidays to do even better things next year, I wish you all the best and to stay healthy. https://imgur.com/a/5rSXMTe As usual, all the sources are in GH [1]. Merry Christmas! Carlos. [1]: https://github.com/ccamacho/tripleo-graphics/tree/master/x-mas/2020

1 0

[blazar] No IRC meetings during holiday season
by Pierre Riteau 21 Dec '20

21 Dec '20

Hello, I am on leave for the end of the year, so there will be no IRC meetings for the Blazar project until the week of January 4. I wish everyone a very happy holiday! Pierre Riteau (priteau)

1 0

[keystone] No weekly meeting Dec 22
by Nikolla, Kristi 21 Dec '20

21 Dec '20

Hi all, I won't be able to host the weekly IRC team meeting for keystone tomorrow. Best, Kristi Nikolla

1 0

Re: [placement] Train upgrade warning
by Seth Tunstall 21 Dec '20

21 Dec '20

Hello, As another person has mentioned (Chanu Romain), I am attempting an upgrade from Queens to Train. I have successfully upgraded Keystone and Glance, but while trying to separate Placement's databases out from Nova, I run into the the same issue with placement-status upgrade check: # placement-status upgrade check +-----------------------------------------------------------------+ | Upgrade Check Results | +-----------------------------------------------------------------+ | Check: Missing Root Provider IDs | | Result: Success | | Details: None | +-----------------------------------------------------------------+ | Check: Incomplete Consumers | | Result: Warning | | Details: There are -3511 incomplete consumers table records for | | existing allocations. Run the "placement-manage db | | online_data_migrations" command. | +-----------------------------------------------------------------+ Likewise, running placement-manage db online_data_migrations does nothing. I am running this placement migration on a dev cluster which has had the databases from a Prod cluster restored to it. This has worked well for me previously as a way to run through upgrades without touching the live system. Thanks very much, Seth

3 10

[tc] weekly update
by Mohammed Naser 21 Dec '20

21 Dec '20

Hi everyone, Here's an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. # Patches ## Open Reviews - Add glance-tempest-plugin to Glance https://review.opendev.org/c/openstack/governance/+/767666 - Remove Karbor project team https://review.opendev.org/c/openstack/governance/+/767056 - Add Resolution of TC stance on the OpenStackClient https://review.opendev.org/c/openstack/governance/+/759904 ## Project Updates - Deprecate openstack-ansible-galera_client role https://review.opendev.org/c/openstack/governance/+/765784 ## General Changes - Improve check-review-status https://review.opendev.org/c/openstack/governance/+/766249 - Clarify impact on releases for SIGs https://review.opendev.org/c/openstack/governance/+/752699 # Other Reminders - Due to the upcoming holidays we are skipping the next TC meetings on Dec 24th and Dec 31st. We will resume with our regular schedule on Thursday January 7th. Thanks for reading! Happy Holidays! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc.

1 0

[all] Our next release name - OpenStack Xena!
by Sean McGinnis 21 Dec '20

21 Dec '20

Hello everyone! The naming poll is done [0], the legal vetting is complete, and we now have our X release name. All hail OpenStack Xena! Observant readers will note that Xena was not the 1st place name from the poll. After review for legal and copyright concerns, it was determined there would be conflicts with the top place Xanadu. Apologies to the Olivia Newton John fans out there. Xena was determined to be safe for our use, and was a close second. More details about the release naming process can be found on the governance page [1]. Please consider getting involved for the Y release naming. Thanks! Sean [0] https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_7e6e96070af39fe7 [1] https://governance.openstack.org/tc/reference/release-naming.html

1 0

New Openstack Deployment questions
by Thomas Wakefield 21 Dec '20

21 Dec '20

OpenStack deployment questions: If you were starting a new deployment of OpenStack today what OS would you use, and what tools would you use for deployment? We were thinking CentOS with Kayobe, but then CentOS changed their support plans, and I am hesitant to start a new project with CentOS. We do have access to RHEL licensing so that might be an option. We have also looked at OpenStack-Ansible for deployment. Thoughts? Thanks in advance. -Tom

9 13

OOO First half on 22 December
by Abhishek Kekane 21 Dec '20

21 Dec '20

Hi Team, I will be out in the first half tomorrow for personal reasons, and will resume work around 1400 UTC. Thanks & Best Regards, Abhishek Kekane

1 1

[tripleo][centosLinux8][podman-1.6] during undercloud deployment missing podman 1.6 cause we have podman 2.0
by Ruslanas Gžibovskis 21 Dec '20

21 Dec '20

Hi all. Somewhere there is a mismatch. When deploying fresh undercloud I get error ./builddir/install-undercloud.log:2020-12-21 11:07:02,763 p=30959 u=root n=ansible | 2020-12-21 11:07:02.762141 | 52540000-0011-172f-d7e6-000000000826 | FATAL | ensure podman and deps are installed | remote-u | error={"changed": false, "failures": ["podman-1.6.4 All matches were filtered out by modular filtering for argument: podman-1.6.4"], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []} ./builddir/install-undercloud.log: "podman-1.6.4" ./builddir/install-undercloud.log: "podman-1.6.4 All matches were filtered out by modular filtering for argument: podman-1.6.4" ./builddir/install-undercloud.log: "podman-1.6.4" so I found a file and updated with version found in repos: [stack@remote-u ~]$ cat /usr/share/ansible/roles/tripleo_podman/vars/redhat.yml --- _tripleo_podman_packages: - podman-2.0.5 _tripleo_buildah_packages: - buildah-1.15.1 _tripleo_podman_purge_packages: - docker - docker-ce [stack@remote-u ~]$ -- Ruslanas Gžibovskis +370 6030 7030

2 3