May 2021 - openstack-discuss - lists.openstack.org

[Nova] Meeting time poll
by Artom Lifshitz 18 May '21

18 May '21

Hey all, As discussed during the IRC meeting today, the Red Hat Nova team would like to know if it's possible to shift the IRC meeting to a different day and/or time. This would facilitate our own internal calls, but I want to be very clear that we'll structure our internal calls around upstream, not the other way around. So please do not perceive any pressure to change, this is just a question :) To help us figure this out, I've created a Doodle poll [1]. I believe the regular attendees of the IRC meeting are spread between Central Europe and NA West Coast, so I've tried to list times that kinda make sense in both of those places, with a couple of hours on each side as a safety margin. Please vote on when you'd like the Nova IRC meeting to take place. Ignore the actual dates (like May 10), the important bits are the days of the week (Monday, Tuesday, etc). This is obviously a recurring meeting, something that Doodle doesn't seem to understand. I've not included Mondays and Wednesdays in the list of possibilities, as they would not work for Red Hat Nova. You can also vote to keep the status quo :) The times are listed in UTC, like the current meeting time, so unfortunately you have to be mindful of the effects of daylight savings time :( Thanks in advance! [1] https://doodle.com/poll/45ptnyn85iuw7pxz

1 2

Restart cinder-volume with Ceph rdb
by ManuParra 18 May '21

18 May '21

Dear OpenStack community, I have encountered a problem a few days ago and that is that when creating new volumes with: "openstack volume create --size 20 testmv" the volume creation status shows an error. If I go to the error log detail it indicates: "Schedule allocate volume: Could not find any available weighted backend". Indeed then I go to the cinder log and it indicates: "volume service is down - host: rbd:volumes@ceph-rbd”. I check with: "openstack volume service list” in which state are the services and I see that indeed this happens: | cinder-volume | rbd:volumes@ceph-rbd | nova | enabled | down | 2021-04-29T09:48:42.000000 | And stopped since 2021-04-29 ! I have checked Ceph (monitors,managers, osds. etc) and there are no problems with the Ceph BackEnd, everything is apparently working. This happened after an uncontrolled outage.So my question is how do I restart only cinder-volumes (I also have cinder-backup, cinder-scheduler but they are ok). Thank you very much in advance. Regards.

7 17

[stein][neutron] l3 agent error
by Ignazio Cassano 17 May '21

17 May '21

Hello Guys, I've just upgraded from openstack queens to rocky and then to stein on centos 7. In my configuration I have router high availability. After the upgrade and rebooting each controller one by one I get the following errors on all my 3 controllers under /var/log/neutron/l3-agent.log http://paste.openstack.org/show/805407/ If I run openstack router show for for one of uuid in the log: http://paste.openstack.org/show/805408/ Namespace for router in present on all 3 controllers. After the controllers reboot,some routers lost their routing tables, but restarting l3-agent they went ok. Is it possible router ha stopped working? Any idea,please ? Ignazio

2 3

[wallaby] patch needed for neutron-linuxbridge-agent: arp_protect.py
by tjoen 17 May '21

17 May '21

neutron-18.0.0 tenacity-7.0.0 Python-3.9.4 neutron-linuxbridge-agent exits with AttributeError: 'FileNotFoundError' object has no attribute 'returncode' neutron/plugins/ml2/drivers/linuxbridge/agent/arp_protect.py: .. @tenacity.retry( wait=tenacity.wait_exponential(multiplier=0.02), retry=tenacity.retry_if_exception(lambda e: e.returncode == 255), reraise=True ) .. Not a python programmer. But I guess it has something to do with Python39

2 2

[magnum] User meeting at 9:00 UTC Tuesday 18 May
by Bharat Kunwar 17 May '21

17 May '21

Hi all, We are planning to meet at 9:00 UTC Tuesday 18 May on #openstack-magnum (freenode). If you have anything pressing you’d like to discuss, please add to the agenda: https://etherpad.opendev.org/p/magnum-weekly-meeting <https://etherpad.opendev.org/p/magnum-weekly-meeting>. Cheers Bharat

1 1

[tripleo] TripleO CI Summary: Unified Sprint 43
by Bhagyashri Shewale 17 May '21

17 May '21

Greetings, The TripleO CI team has just completed **Unified Sprint 43** (Apr 8 thru to Apr 28 wed 2021). The following is a summary of completed work during this sprint cycle: - Successfully deployed promoter server for all c8 and c7 releases using new code (next gen) promoter code <https://hackmd.io/team/rh-openstack-ci?nav=search&q=promoter> - Continue to adopt new changes and improvements in the promoter : - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32135 - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32058 - Reduce our upstream resource usage with zuul jobs including optimizations across the zuul layouts in all the tripleo-* repos - https://review.opendev.org/q/topic:tripleo-ci-reduce - Removing Rocky jobs (EOL) - https://review.opendev.org/q/topic:%22tripleo-ci-reduce-rocky%22 - https://review.rdoproject.org/r/q/topic:%22tripleo-ci-reduce-rocky%22 - Content provider jobs across all branches - Wallaby branching -Component / integration rdo pipelines for wallaby - https://hackmd.io/2sxlx1XzTa-Te47_zLv42Q - Repo sanity check as part of the integration lines - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/785040 - Correct the tagging in copy-quay tool - the copy and tagging was one single operation, now copy command is copying every hour all the hashes available and tag is running separately now collecting the current-tripleo tag directly from the delorean api. - copy-quay script: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32478 - Added all the monitoring work items for wallaby in Grafana: - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32890 - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33125 - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33238 - Python version of cloud infra cleanup script - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/31410 The planned work for the next sprint and leftover previous sprint work as following: - Deploy the promoter server for upstream promotions on vexxhost - Dependency Pipeline updates: - Update RHEL_U_Next to handle RHEL8.4 + ct3.0 + rhos-release scenario - elastic-recheck containerization - https://hackmd.io/dmxF-brbS-yg7tkFB_kxXQ - Openstack health for tripleo - https://hackmd.io/HQ5hyGAOSuG44Le2x6YzUw - Tripleo-repos spec and implementation - Adds tripleo-get-hash module get tripleo-ci hash info from tag - https://review.opendev.org/c/openstack/tripleo-ci/+/784392 - https://hackmd.io/v2jCX9RwSeuP8EEFDHRa8g?view - https://review.opendev.org/c/openstack/tripleo-specs/+/772442 - Tempest skiplist: Started to work on tempest-skiplist allowed list of tests (no patches yet) - Patrole stable release- Started to work on it - Ensure pip installs do not impact the tripleo deployment - https://review.opendev.org/q/topic:%22tqe_requires_cleanup%22 The Ruck and Rover for this sprint are Amol Kahat(akahat) and Pooja Jadhav(pojadhav). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes to be tracked in hackmd. Thanks, Bhagyashri Shewale

1 0

[all][tc] What's happening in Technical Committee: summary 14th May, 21: Reading: 5 min
by Ghanshyam Mann 17 May '21

17 May '21

Hello Everyone, Here is last week's summary of the Technical Committee activities. 1. What we completed this week: ========================= Project updates: ------------------- ** puppet-glare is retired[1] Other updates: ------------------ ** Merged the 'Y' release naming process schedule[2]. 2. TC Meetings: ============ * TC held this week meeting on Thursday; you can find the full meeting logs in the below link: - http://eavesdrop.openstack.org/meetings/tc/2021/tc.2021-05-13-15.00.log.html * We will have next week's meeting on May 20th, Thursday 15:00 UTC[3]. 3. Activities In progress: ================== TC Tracker for Xena cycle ------------------------------ TC is using the etherpad[4] for Xena cycle working item. We will be checking and updating the status biweekly in the same etherpad. Open Reviews ----------------- * Two open reviews for ongoing activities[5]. Starting the 'Y' release naming process --------------------------------------------- * Y release naming process is started[6]. Nomination is open until June 10th feel free to propose names in below wiki ** https://wiki.openstack.org/wiki/Release_Naming/Y_Proposals Others --------- * Replacing ATC terminology with AC[7]. As UC is merged into TC, this is to include the AUC into ATC so that they can be eligible for TC election voting. We are having a good amount of discussion on Gerrit, feel free to review the patch if you have any points regarding this. 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[8]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [9] 3. Office hours: The Technical Committee offers a weekly office hour every Tuesday at 0100 UTC [10] 4. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://review.opendev.org/c/openstack/governance/+/790582 [2] https://review.opendev.org/c/openstack/governance/+/789385 [3] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [4] https://etherpad.opendev.org/p/tc-xena-tracker [5] https://review.opendev.org/q/project:openstack/governance+status:open [6] http://lists.openstack.org/pipermail/openstack-discuss/2021-May/022383.html [7] https://review.opendev.org/c/openstack/governance/+/790092 [8] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [9] http://eavesdrop.openstack.org/#Technical_Committee_Meeting [10] http://eavesdrop.openstack.org/#Technical_Committee_Office_hours -gmann

1 0

[Octavia][Victoria] No service listening on port 9443 in the amphora instance
by Luke Camilleri 17 May '21

17 May '21

Hi there, i am trying to get Octavia running on a Victoria deployment on CentOS 8. It was a bit rough getting to the point to launch an instance mainly due to the load-balancer management network and the lack of documentation (https://docs.openstack.org/octavia/victoria/install/install.html) to deploy this oN CentOS. I will try to fix this once I have my deployment up and running to help others on the way installing and configuring this :-) At this point a LB can be launched by the tenant and the instance is spawned in the Octavia project and I can ping and SSH into the amphora instance from the Octavia node where the octavia-health-manager service is running using the IP within the same subnet of the amphoras (172.16.0.0/12). Unfortunately I keep on getting these errors in the log file of the worker log (/var/log/octavia/worker.log): 2021-05-05 01:54:49.368 14521 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.4.46', p ort=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f83e0181550>: Failed to establish a new connection: [Errno 111] Connection ref used',)) 2021-05-05 01:54:54.374 14521 ERROR octavia.amphorae.drivers.haproxy.rest_api_driver [-] Connection retries (currently set to 120) exhausted. The amphora is unavailable. Reason: HTTPSConnectionPool(host='172.16 .4.46', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f83e0181550>: Failed to establish a new connection: [Errno 111] Conne ction refused',)) 2021-05-05 01:54:54.374 14521 ERROR octavia.controller.worker.v1.tasks.amphora_driver_tasks [-] Amphora compute instance failed to become reachable. This either means the compute driver failed to fully boot the instance inside the timeout interval or the instance is not reachable via the lb-mgmt-net.: octavia.amphorae.driver_exceptions.exceptions.TimeOutException: contacting the amphora timed out obviously the instance is deleted then and the task fails from the tenant's perspective. The main issue here is that there is no service running on port 9443 on the amphora instance. I am assuming that this is in fact the amphora-agent service that is running on the instance which should be listening on this port 9443 but the service does not seem to be up or not installed at all. To create the image I have installed the CentOS package "openstack-octavia-diskimage-create" which provides the utility disk-image-create but from what I can conclude the amphora-agent is not being installed (thought this was done automatically by default :-( ) Can anyone let me know if the amphora-agent is what gets queried on port 9443 ? If the agent is not installed/injected by default when building the amphora image? The command to inject the amphora-agent into the amphora image when using the disk-image-create command? Thanks in advance for any assistance

2 10

[neutron][stein] l3 agent errors
by Ignazio Cassano 16 May '21

16 May '21

Hello Guys, I've just upgraded from openstack queens to rocky and then to stein on centos 7. In my configuration I have router high availability. After the upgrade I get the following errors on all my 3 controllers under /var/log/neutron/l3-agent.log 2021-05-15 19:04:28.599 13219 ERROR neutron.agent.linux.external_process [-] ip_monitor for router with uuid 8c137f17-52e6-47f9-bcaa-0eac94a9acd7 not found. The process should not have died 2021-05-15 19:04:28.600 13219 WARNING neutron.agent.linux.external_process [-] Respawning ip_monitor for uuid 8c137f17-52e6-47f9-bcaa-0eac94a9acd7 2021-05-15 19:04:29.905 13219 ERROR neutron.agent.linux.external_process [-] ip_monitor for router with uuid 303c88e4-1c94-4ebc-b592-91063b371db9 not found. The process should not have died 2021-05-15 19:04:29.905 13219 WARNING neutron.agent.linux.external_process [-] Respawning ip_monitor for uuid 303c88e4-1c94-4ebc-b592-91063b371db9 2021-05-15 19:04:31.267 13219 ERROR neutron.agent.linux.external_process [-] ip_monitor for router with uuid 103f3588-a775-4bca-b7a5-093a05068f64 not found. The process should not have died 2021-05-15 19:04:31.268 13219 WARNING neutron.agent.linux.external_process [-] Respawning ip_monitor for uuid 103f3588-a775-4bca-b7a5-093a05068f64 2021-05-15 19:04:32.684 13219 ERROR neutron.agent.linux.external_process [-] ip_monitor for router with uuid c6136e08-16a3-4d56-9440-59a7fe90d0b0 not found. The process should not have died 2021-05-15 19:04:32.685 13219 WARNING neutron.agent.linux.external_process [-] Respawning ip_monitor for uuid c6136e08-16a3-4d56-9440-59a7fe90d0b0 2021-05-15 19:04:34.140 13219 ERROR neutron.agent.linux.external_process [-] ip_monitor for router with uuid 9abaad3a-2439-4036-8611-5f93a8a3501f not found. The process should not have died 2021-05-15 19:04:34.140 13219 WARNING neutron.agent.linux.external_process [-] Respawning ip_monitor for uuid 9abaad3a-2439-4036-8611-5f93a8a3501f Getting for example the last uuid if I execute "openstack router show openstack router show 9abaad3a-2439-4036-8611-5f93a8a3501f +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2020-12-17T10:11:24Z | | description | | | distributed | False | | external_gateway_info | None | | flavor_id | None | | ha | True | | id | 9abaad3a-2439-4036-8611-5f93a8a3501f | | interfaces_info | [{"subnet_id": "a60b631d-d13c-4d02-a51b-2c86c0b38614", "ip_address": "192.168.201.1", "port_id": "08120fb0-be86-47a0-afd6-932801a2d194"}, {"subnet_id": "0ffcb170-66bb-4e65-9df0-80052deb1299", "ip_address": "169.254.192.11", "port_id": "3aa676aa-91cc-43c4-87ea-3b93340950ad"}, {"subnet_id": "0ffcb170-66bb-4e65-9df0-80052deb1299", "ip_address": "169.254.192.21", "port_id": "d5c0cff2-363d-470a-b77c-155d06e2654b"}, {"subnet_id": "0ffcb170-66bb-4e65-9df0-80052deb1299", "ip_address": "169.254.192.3", "port_id": "dab50b1a-d720-410a-b358-20c53a097bd7"}, {"subnet_id": "ae117b0e-fce9-42ca-88b3-a32ba239d795", "ip_address": "192.168.96.59", "port_id": "e49945aa-3e71-4f38-8c76-1c246cf18ff6"}] | | location | Munch({'project': Munch({'domain_name': None, 'domain_id': None, 'name': None, 'id': u'07573f587ceb4d9da007cec4ebc5dcf8'}), 'cloud': '', 'region_name': '', 'zone': None}) | | name | InternetGateway-a820a62fae-avz714-3-router | | project_id | 07573f587ceb4d9da007cec4ebc5dcf8 | | revision_number | 11 | | routes | destination='0.0.0.0/0', gateway='192.168.96.60' | | | destination='192.168.200.0/24', gateway='192.168.96.60' | | | destination='192.168.202.0/24', gateway='192.168.96.58' | | | destination='192.168.203.0/24', gateway='192.168.96.57' | | status | ACTIVE | | tags | | | updated_at | 2020-12-18T15:22:57Z | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Namespace for router exists in all 3 controllers. Any idea, please ? Ignazio

1 0

[victoria] Windows Server 2008 SP2 - No bootble device
by SSelf＠performair.com 15 May '21

15 May '21

All; I'm having some trouble getting a Windows Server 2008 SP2 image to boot. On instance start I am greeted with the following: Booting from Hard Disk... Boot failed: not a bootable disk No bootable device. I've followed the instructions, referenced below, with a minor exception; the "Red Hat VirtIO SCSI controller" driver was installed after the OS had been installed as the Pre-installation Environment had troubles recognizing them (yay 2008?). Any recommendations would be welcome. Reference: https://superuser.openstack.org/articles/how-to-deploy-windows-on-openstack/ Thank you, Stephen Self IT Manager Perform Air International Inc. sself(a)performair.com www.performair.com

1 0