October 2021 - openstack-discuss - lists.openstack.org

[nova][placement] Yoga PTG summary
by Sylvain Bauza 26 Oct '21

26 Oct '21

Well, it's a try given we discussed for 4 days and it could be a large summary ;) You can see all the notes in a read-only etherpad here : https://etherpad.opendev.org/p/r.e70aa851abf8644c29c8abe4bce32b81 ### Cross-project discussions # Cyborg cross-project discussion with Nova We agreed on adding a OWNED_BY_NOVA trait for all Resource Providers creating by Nova so Cyborg would provide their own OWNED_BY_CYBORG trait for knowing which inventories are used by either Nova or Cyborg. Cyborg contributors need to modify https://review.opendev.org/c/openstack/nova-specs/+/780452 We also agreed on the fact that https://blueprints.launchpad.net/nova/+spec/cyborg-suspend-and-resume is a specless blueprint. # Oslo cross-project discussion with Nova gmann agreed on providing a new flag for oslopolicy-sample-generator for adding deprecated rules in the generated policy file. # RBAC popup team discussion with Nova Eventually, we found no consensus for this topic as there were some left open questions about system-scope. FWIW, the popup team then discussed on Friday with the TC about this, so please look at the TC etherpad if you want to know more. A side impact is about https://review.opendev.org/c/openstack/nova-specs/+/793011 which is now punted until we figure out a good path. # Neutron cross-project discussion with Nova Again, about RBAC for external events api interaction, we discussed about the scopes and eventually punted the discussion. About specific events related to Neutron backends, ralonso accepted to provide a documentation explaining what backends sends which events, and we accepted to merge https://review.opendev.org/c/openstack/nova/+/813419 as a short-term solution while we would like to get a long-term solution by having Neutron providing the event information by the port binding information. About testing move operations, we agreed on continuing to have a ML2/OVS multinode job. During another Nova session, we also agreed on changing libvirt to directly unplug (not unbind) ports during VM shutdown. # Interop cross-project disussion with Nova We agreed on reviewing https://review.opendev.org/c/openinfra/interop/+/811049/3/guidelines/2021.1… # Cinder cross-project discussion with Nova We discussed about https://blueprints.launchpad.net/nova/+spec/volume-backed-server-rebuild and we said the workflow should be something like user > nova > cinder > nova. We also discussed about some upgrade questions for this one, but we eventually agreed on it. We also discussed about devstack-plugin-nfs gate issues and how to contact the nova events API for resize. # Manila integration with libvirt driver The first spec looks promising https://review.opendev.org/c/openstack/nova-specs/+/813180 There were discussions about cloud-init and the Ironic case, but we agree on the smaller scope for the Yoga release that's proposed in the spec. ### Nova specific topics ### # Xena retrospective We agreed on stopping to have an Asian-friendly meeting timeslot once per month as unfortunately, no contributors went to meetings when we had them. We also agreed on modifying the review-priority label for abling contributors to use it too, but we first need to provide a documentation explaining it before we change Gerrit. # Continuing to use Storyboard for Placement or not ? We eventually said we would look at how create a script for moving Storyboard (SB) stories to Launchpad (as either features or bugs) but we still want to continue verifying whether contributors use Storyboard for Placement bugs or feature requests and bauzas accepted to provide visibility on Placement SB stories during the weekly meeting. gmann also agreed on asking contributors to move off the #openstack-placement IRC channel to #openstack-nova so we would delete this IRC channel eventually during this cycle. # Yoga release deadlines for Nova No real change in deadlines compared to the Xena timeframe. We agreed on having two spec review days, one around mid-Nov (before yoga-1) and one around mid-Dec (before Christmas period) in order to prioritize implementation reviews after Jan 1st even if we can continue to review specs until yoga-2. # python version pinning with tox We agreed on the fact this is a pain for developers. We then had a consensus on modifying tox to accept multiple python versions automatically with https://review.opendev.org/c/openstack/nova/+/804292 so it would fix the issue. # Nova DB issues with DB coalation We agreed on the fact it's a problem, so we'll document the fact that Nova APIs are case-insensitive at the moment even if Python is case-sensitive, which creates problems. What we propose in order to fix this is to provide a nova-manage db upgrade command that would modify the DB tables to use COLLATE utf8_bin but we also agree we can't ask operators to migrate by one cycle and we accept the fact this command could be there for a while. # SQLAlchemy 2.0 The wave is coming and we need contributors to help us change what we have in our nova DB modules to no longer use the deprecated calls. This looks a low-hanging-fruit and I'll try to find some contributor for this. # Bumping minimum microversion from v2.1 No, we said no because $users use it. # Unified limits We agreed on providing a read-only API for knowing the limits but *not* providing proxy APIs for setting limits *as a first attempt*. We prefer operators to come back with usecases and feedback from their use of Unified Limits 1.0 before we start drafting some proxy API to Keystone. Also, we agreed on *not* having config-driven quotas. # Central vncproxy in a multiple-cells environment We understand the usecase which is very specific and we accept to create a central vncproxy service that would proxy calls to the cell-related vncproxy service but this is not a pattern we want to follow for every cell-specific nova service. # Move instances between projects. Well, we totally get the usecase but we absolutely lack of resources in order to work on this very large effort that would span multiple services. # Nova service healthchecks We agreed on providing a way for monitoring tools to ping Nova services for healthness thru http or unix socket, from every service that would return a status based on cached data. A spec has to be written. # Zombie Resource Providers no longer corresponding to Nova resources Thanks to the OWNED_BY_NOVA trait we agreed when discussing with the Cyborg team, we could find a way to know the ResourceProviders owned by Nova that are no longer in use and we could consequently delete them, or warn the operator if existing allocations are present. # NUMA balancing This is definitely a bug we will fix by providing a workaround config option that will let operators define the packing or spreading strategy they want for NUMA cell stacking (or not). # Deprecation of the novaclient shell command Yes, now that the SDK is able to negociate, we can deprecate the novaclient CLI. More investigation work has to be done in order to know whether we can also deprecate the novaclient library itself. # Integration with Off-Path Network Backends Lots of technicalities with this very large spec https://review.opendev.org/c/openstack/nova-specs/+/787458. Long story short, we asked the proposer to add a few more details in the spec about upgrade scenarios, move operations and testing, but we also told that we can't accept this spec until the Neutron one lands as there are some usage from the extended Neutron APIs that are proposed in the Nova spec. # 'pc' and 'q35' machine types We agreed on changing the default machine type to 'q35' but we also agreed on *not* deprecating the 'pc' type. Some documentation has to be written in order to explain the intent of the migration. # Nova use of privsep sean-k-mooney agreed on working on a patch to remove usage of CAP_DAC_OVERRIDE and on another patch to provide the new capabilities for the monolith privsep context (a few other topics were discussed but I skipped them from my summary as they're not significant for either operators or other contributors but a very few people - don't get me wrong, I like them but I don't wanna add more verbosity to an already large summary) ### Nova painpoints (taken from https://etherpad.opendev.org/p/pain-point-elimination) # Ironic hashring failure recovery That's a latent bug but we don't want the Ironic virt driver to modify the host value of every instance. We'd rather prefer some operator action in order to communicate Nova it has to shuffle a few things. More brainstorm has honestly to be done with this as we haven't clearly drafted a designed solution yet. # Problems with shelve, unshelve and then shelve back Well, this is a nasty bug and we need to fix it, agreed. We also have a testing gap we need to close. # Naming cinder volumes after nova instance name We said yes, why not. We also considered that 'delete on terminate' has to change so it does delete the volume when you delete the instance (for a boot-for-volume case) # Orphaned instances due to underlying network issues We agreed on the fact it would be nice to provide a tool for knowing such orphans and we also think it's import for instance force-delete API call to complete successfully in such case. # reminiscent guests on a recovered compute node while instance records were purged Well, we should avoid to purge instance records from the Nova DB if we are still unable to correctly delete the compute bits unless the operator explicitly wants to (in the case of a non-recoverable compute for example). A potential solution can be to add a config flag to *not* archive deleted rows on instances that are running on down computes. # Nova APIs leaking hypervisor hardware details No, we don't want this but we can try to add more traits for giving more visibility on a case-by-case basis # RabbitMQ replacement Well, we're unfortunately lacking resources on this. We recommend the community to investigate on use of a NATS backend for oslo.messaging # Placement strictness with move operations We agree this is a bug that needs to be fixed. Okay, if you reach this point, you're very brave. Kudos to you. I don't really want to reproduce this exercice often, but I just hope it helps you summarizing a thousand-line large etherpad. -Sylvain

2 2

Octavia loadbalancer status offline
by Md. Hejbul Tawhid MUNNA 26 Oct '21

26 Oct '21

Hi, We have installed openstack ussuri version from ubuntu universe repository. We have installed octavia 6.2.0 version. after creating loadbalancer , listener and pool all are offline. but the LB operation is working as expected. changing the pool member is also working. octavia is installed in compute node. 5555 is listening and allowed in iptables (iptables -I INPUT -i o-hm0 -p udp --dport 5555 -j ACCEPT). amphora to octavia-worker(172.16.0.2) is reachable. Any idea to troubleshoot this issue Please find the log from octavia-worker ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 2021-10-25 18:15:13.482 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540e192be0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:18.490 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c0ddd00>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:23.507 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c0ddd90>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:28.511 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c0ddbb0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:33.521 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c064190>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:38.529 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540d111790>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:43.540 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec7a3670>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:48.549 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec74d0d0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:53.554 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec75a7f0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:58.561 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec75a6a0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:16:04.707 1192307 INFO octavia.controller.worker.v1.tasks.database_tasks [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Mark ALLOCATED in DB for amphora: c661b828-1690-4866-8152-f745c43e0977 with compute id c9133819-b8e0-42d6-9544-bf83e3ad4b3f for load balancer: c0bd3e21-6983-40c9-8713-859194496b37 2021-10-25 18:16:40.660 1192307 INFO octavia.controller.worker.v1.tasks.database_tasks [req-78e78f29-3bdb-4b12-ae76-b8daa4926c09 - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Mark ACTIVE in DB for load balancer id: c0bd3e21-6983-40c9-8713-859194496b37 2021-10-25 18:16:44.317 1192307 INFO octavia.controller.queue.v1.endpoints [-] Creating listener 'cc45192d-de70-4d59-857b-ac23c4fc8d07'... 2021-10-25 18:16:44.325 1192307 WARNING octavia.controller.worker.v1.controller_worker [-] Failed to fetch listener cc45192d-de70-4d59-857b-ac23c4fc8d07 from DB. Retrying for up to 60 seconds. 2021-10-25 18:17:35.375 1192307 INFO octavia.controller.queue.v1.endpoints [-] Creating pool '9d23855f-d849-4ad9-9de1-66ab5cd268eb'... 2021-10-25 18:17:35.382 1192307 WARNING octavia.controller.worker.v1.controller_worker [-] Failed to fetch pool 9d23855f-d849-4ad9-9de1-66ab5cd268eb from DB. Retrying for up to 60 seconds. 2021-10-25 18:18:02.814 1192307 INFO octavia.controller.queue.v1.endpoints [-] Creating member '29bb41e5-457c-43ba-9149-5af55e73fe38'... 2021-10-25 18:18:02.825 1192307 WARNING octavia.controller.worker.v1.controller_worker [-] Failed to fetch member 29bb41e5-457c-43ba-9149-5af55e73fe38 from DB. Retrying for up to 60 seconds. Regards, Munna

3 4

Re: Openstack magnum
by Vikarna Tathe 26 Oct '21

26 Oct '21

Hi Yasemin, You can find all the stable releases in the below link https://builds.coreos.fedoraproject.org/browser?stream=stable&arch=x86_64 Vikarna On Sun, 24 Oct 2021 at 17:28, Yasemin DEMİRAL (BILGEM BTE) < yasemin.demiral(a)tubitak.gov.tr> wrote: > Hi, > > How can I dowloand fcos 33? I can't find any link for dowloanding it. > > *Yasemin DEMİRAL* > <http://www.tubitak.gov.tr/tr/icerik-sorumluluk-reddi> > > Senior Researcher at TUBITAK BILGEM B3LAB > > Safir Cloud Scrum Master > > > ------------------------------ > *Kimden: *"Vikarna Tathe" <vikarnatathe(a)gmail.com> > *Kime: *"Ammad Syed" <syedammad83(a)gmail.com> > *Kk: *"openstack-discuss" <openstack-discuss(a)lists.openstack.org> > *Gönderilenler: *19 Ekim Salı 2021 16:23:20 > *Konu: *Re: Openstack magnum > > Hi Ammad, > Thanks!!! It worked. > > On Tue, 19 Oct 2021 at 15:00, Vikarna Tathe <vikarnatathe(a)gmail.com> > wrote: > >> Hi Ammad, >> Yes, fcos34. Let me try with fcos33. Thanks >> >> On Tue, 19 Oct 2021 at 14:52, Ammad Syed <syedammad83(a)gmail.com> wrote: >> >>> Hi, >>> >>> Which fcos image you are using ? It looks like you are using fcos 34. >>> Which is currently not supported. Use fcos 33. >>> >>> On Tue, Oct 19, 2021 at 2:16 PM Vikarna Tathe <vikarnatathe(a)gmail.com> >>> wrote: >>> >>>> Hi All, >>>> I was able to login to the instance. I see that kubelet service is in >>>> activating state. When I checked the journalctl, found the below. >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Oct 19 05:18:34 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: >>>> Started Kubelet via Hyperkube (System Container).Oct 19 05:18:34 >>>> kubernetes-cluster-6cdrblcpckny-master-0 bash[6521]: Error: statfs >>>> /sys/fs/cgroup/systemd: no such file or directoryOct 19 05:18:34 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: Main >>>> process exited, code=exited, status=125/n/aOct 19 05:18:34 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: >>>> Failed with result 'exit-code'.Oct 19 05:18:44 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: >>>> Scheduled restart job, restart counter is at 18.Oct 19 05:18:44 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: Stopped Kubelet via >>>> Hyperkube (System Container).* >>>> >>>> Executed the below command to fix this issue. >>>> *mkdir -p /sys/fs/cgroup/systemd* >>>> >>>> >>>> Now I am getiing the below error. Has anybody seen this issue. >>>> >>>> >>>> >>>> *failed to get the kubelet's cgroup: mountpoint for cpu not found. >>>> Kubelet system container metrics may be missing.failed to get the container >>>> runtime's cgroup: failed to get container name for docker process: >>>> mountpoint for cpu not found. failed to run Kubelet: mountpoint for not >>>> found* >>>> >>>> On Mon, 18 Oct 2021 at 14:09, Vikarna Tathe <vikarnatathe(a)gmail.com> >>>> wrote: >>>> >>>>> >>>>>> Hi Ammad, >>>>>> Thanks for responding. >>>>>> >>>>>> Yes the instance is getting created, but i am unable to login >>>>>> though i have generated the keypair. There is no default password for this >>>>>> image to login via console. >>>>>> >>>>>> openstack server list >>>>>> >>>>>> +--------------------------------------+--------------------------------------+---------+------------------------------------+----------------------+----------+ >>>>>> | ID | Name >>>>>> | Status | Networks | Image >>>>>> | Flavor | >>>>>> >>>>>> +--------------------------------------+--------------------------------------+---------+------------------------------------+----------------------+----------+ >>>>>> | cf955a75-8cd2-4f91-a01f-677159b57cb2 | >>>>>> k8s-cluster-01-2nyejxo3hyvb-master-0 | ACTIVE | private1=10.100.0.39, >>>>>> 10.14.20.181 | fedora-coreos-latest | m1.large | >>>>>> >>>>>> >>>>>> ssh -i id_rsa core(a)10.14.20.181 >>>>>> The authenticity of host '10.14.20.181 (10.14.20.181)' can't be >>>>>> established. >>>>>> ECDSA key fingerprint is >>>>>> SHA256:ykEMpwf79/zTMwcELDSI0f66Sxbri56ovGJ+RRwKXDU. >>>>>> Are you sure you want to continue connecting (yes/no/[fingerprint])? >>>>>> yes >>>>>> Warning: Permanently added '10.14.20.181' (ECDSA) to the list of >>>>>> known hosts. >>>>>> core(a)10.14.20.181: Permission denied >>>>>> (publickey,gssapi-keyex,gssapi-with-mic). >>>>>> >>>>>> On Mon, 18 Oct 2021 at 14:02, Ammad Syed <syedammad83(a)gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> Can you check if the master server is deployed as a nova instance ? >>>>>>> if yes, then login to the instance and check cloud-init and heat agent logs >>>>>>> to see the errors. >>>>>>> >>>>>>> Ammad >>>>>>> >>>>>>> On Mon, Oct 18, 2021 at 12:03 PM Vikarna Tathe < >>>>>>> vikarnatathe(a)gmail.com> wrote: >>>>>>> >>>>>>>> Hello All, >>>>>>>> I am trying to create a kubernetes cluster using magnum. Image: >>>>>>>> fedora-coreos. >>>>>>>> >>>>>>>> >>>>>>>> The stack gets stucked in CREATE_IN_PROGRESS. See the output below. >>>>>>>> openstack coe cluster list >>>>>>>> >>>>>>>> +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ >>>>>>>> | uuid | name | keypair | >>>>>>>> node_count | master_count | status | health_status | >>>>>>>> >>>>>>>> +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ >>>>>>>> | 9381cd12-af60-40b1-b1c4-d3dcc93e926e | k8s-cluster-01 | ctrlkey | >>>>>>>> 2 | 1 | CREATE_IN_PROGRESS | None | >>>>>>>> >>>>>>>> +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ >>>>>>>> >>>>>>>> openstack stack resource show k8s-cluster-01-2nyejxo3hyvb >>>>>>>> kube_masters >>>>>>>> >>>>>>>> +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >>>>>>>> | Field | Value >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> >>>>>>>> +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >>>>>>>> | attributes | {'refs_map': None, 'removed_rsrc_list': >>>>>>>> [], 'attributes': None, 'refs': None} >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | creation_time | 2021-10-18T06:44:02Z >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | description | >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | links | [{'href': ' >>>>>>>> http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', >>>>>>>> 'rel': 'self'}, {'href': ' >>>>>>>> http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', >>>>>>>> 'rel': 'stack'}, {'href': ' >>>>>>>> http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', >>>>>>>> 'rel': 'nested'}] | >>>>>>>> | logical_resource_id | kube_masters >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | physical_resource_id | 3da2083f-0b2c-4b9d-8df5-8468e0de3028 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | required_by | ['kube_cluster_deploy', >>>>>>>> 'etcd_address_lb_switch', 'api_address_lb_switch', 'kube_cluster_config'] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_name | kube_masters >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_status | CREATE_IN_PROGRESS >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_status_reason | state changed >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_type | OS::Heat::ResourceGroup >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | updated_time | 2021-10-18T06:44:02Z >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> >>>>>>>> +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >>>>>>>> >>>>>>>> Vikarna >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Syed Ammad Ali >>>>>>> >>>>>> -- >>> Regards, >>> >>> Syed Ammad Ali >>> >> >

1 0

kolla-ansible wallaby manila ceph pacific
by wodel youchi 26 Oct '21

26 Oct '21

Hi, Has anyone been successful in deploying Manila wallaby using kolla-ansible with ceph pacific as a backend? I have created the manila client in ceph pacific like this : *ceph auth get-or-create client.manila mon 'allow r' mds 'allow r, allow rw path=/' osd 'allow rw pool=cephfs_data' mgr 'allow rw'* When I deploy, I get this error in manila's log file : Bad target type 'mon-mgr' Any ideas? Regards.

3 7

[neutron] October 2021 - PTG Summary
by Lajos Katona 26 Oct '21

26 Oct '21

Hi, Below is my summary of the Neutron sessions during last week's PTG. Here is the etherpad with the agenda: https://etherpad.opendev.org/p/neutron-yoga-ptg # Day 1 (Monday Oct 18.) ## Xena retrospective (I just list here the topics, not every discussion under them): * Good things: ** Less meetings ** New features ** CI jobs have again meaningful names ** Housekeeping - new payload style callbacks switch finished ** Healthy team - we have people from various companies, doing reviews and proposing patches * Bad / not so good / change it ** CI stability could be better ** lack of people involved in the CI meetings and CI issues ** is there a chance to decrease the number of jobs/backends to be * Actions coming out from the above: ** Having video based team meetings every few weeks. ** Video meeting with screen sharing for CI meeting every few weeks. ** ping people before the meetings ## Should we move some parts of neutron devstack plugin to devstack repo? Slaweq brought this topic to rationalize devstack plugin code between neutron and devstack. As a rule of thumb: "what is used/tested by jobs from other repositories (like tempest) should be in devstack repo and not in neutron's devstack plugin". The bad part is that we should loose review velocity if there's nobody to review in devstack. As a consequence some code should be moved back from devstack to neutron's devstack plugin (for drivers that are not that common now like linuxbridge) ## ML2/OVS -> ML2/OVN migration - presentation (continued on Friday) We got an overview from Jakub (jlibosva) of how the migration tool works now, what is finished and which are the possible directions to improve it. https://docs.google.com/presentation/d/1dCdvoi-Cdbl27AL0wjFJ_51NatOxepNOplm… * Problematic parts: ** steps that require the tool to touch the db (no API currently) ** no rollback ** no CI (at least upstream) ** no devstack based tools (related to the previous one) * Actions ** New job for ovs-to-ovn migration *** Now start with tripleo based job and go for a devstack based one ** Don't create new APIs for job type change and other operations during the migration as it seems there is no other usecase for them. ** Add developer documentation for the migration (preparation, steps, rollback....) ** add automatic rollback # Day 2 (Tuesday Oct. 19.) ## tempest_neutron_plugin - a RedHat downstream tests We got an overview of these tests from Eduardo Olivares. * tests requiring advance images ** move them to neutron-tempest-plugin with special tag and run them periodically ** use ubuntu cloud minimal image * tests requiring multinode topology + capturing traffic on hosts/computes ** These tests use a node to run tempest (tripleo based deployment) ** these tests should be changed to run in upstream CI * Tests that restart nodes / services / containers ** better not to run them in upstream CI ** check if it is possible to migrate some to tobiko ( https://opendev.org/x/tobiko ) * sriov tests (macvtap, direct and direct-physical ports) ** No possibility today to run these upstream. ## Edge session (https://etherpad.opendev.org/p/ecg-ptg-october-2021 ) * Thursday we had a session together with the Designate team where we discussed possible action items related to edge use cases. ## RBAC together with Nova * https://etherpad.opendev.org/p/nova-yoga-ptg l108. - l137. # Day 3 (Wednesday Oct. 20.) ## Job reorganization recap * We need fresh data to see where we still have pain points. * Ongoing CI rationalization items: ** Use Ubuntu minimal image where possible. ** Serialize the run of tests with advanced images (reduce memory usage): *** https://review.opendev.org/c/openstack/tempest/+/807790 ## ryu / os-ken Recently we realized that ryu has some activity to fix bugs that can hit Neutron as well (as we forked ryu as os-ken). * regularly checking ryu bugs and fixes (sync on team meeting) * Do not return to ryu as os-ken is adopted for Openstack workflow with CI, etc... ## Nova-Neutron cross-project session * OVN live-migration (https://etherpad.opendev.org/p/ovn_live_migration ) * RBAC continued (https://etherpad.opendev.org/p/nova-yoga-ptg l144 - l165) * libvirt virt driver does not wait for network-vif-plugged event during hard reboot ** document which backend sends what event and when ** workaround in nova for telling whether we should wait for an event ( https://review.opendev.org/c/openstack/nova/+/813419 ) ** long-term solution has to be Neutron providing event information thru the port binding information ** Improved testing of move operations *** Nova will keep one ML2/OVS multinode job running ** Supporting ports with QoS min bw and min pps in multi segment networks with multiple physnets *** finish the ongoing QoS min pps support *** extend placement to support any-trait queries ## Interop with Martin Kopec * Check which tests should be included to refstack * https://etherpad.opendev.org/p/refstack-test-analysis ## Quick introduction of the Tobiko test tool We got an overview of Tobiko from Frederico Ressi, and how it can be used to test scenarios which are destructive. * https://tobiko.readthedocs.io/en/master/ * http://kaplonski.pl/files/Tobiko_-_OpenStack_upgrade_and_disruptive_testing… * Neutron already has a periodic job with tobiko. # Day 4 (Thursday Oct. 21.) ## OVN and BGP * https://opendev.org/x/ovn-bgp-agent ** not just for neutron-dynamic-routing but for networking-bgpvpn as well. ** ovn-bgp-agent is another backend which could be behind neutron-dynamic-routing, and uses FRR. ** In the future it will be a Neutron stadium project ** need more tests, integration with Neutron and better documentation. * Make neutron-dynamic-routing work with OVN ** Frickler and bcafarel checked if the current jobs and tests can work with OVN. ** more investigation is necessary if it works as is now or some change is needed. ** Redhat and Ericsson help in the maintenance of neutron-dynamic-routing. ## Multi segments and provider networks on a host, with 1 subnet per segment * different from the usual routed network use case where we have one segment available in each rack * WIP patch which was never finished: ** https://review.opendev.org/c/openstack/neutron/+/623115 ## Use openstacksdk as client for Heat * Now Heat uses python-neutronclient for python bindings / client (not the CLI part which is deprecated but the python bindings) ** We collected a buch of projects over than heat which uses python-neutronclient: Nova, Horizon, Rally.... ** Actions: *** Write mail about it to have the attention of the community *** Have a migration plan and document it. *** Have a slot during the team meeting to track progress. ## Designate session The topics coming from Designate team were really helpful to translate the requirements from edge session to action items for Neutron. * https://etherpad.opendev.org/p/octavia-designate-neutron-ptg ** dns resolver is not specified for a subnet, (RFE for OVN as it works only for OVS currently) ** initiate a cross-project documentation effort for setting up an edge site ** initiate cross-project job for testing an edge site # Day 5 (Friday Oct. 22.) ## Continued discussion of neutron-dynamic-routing's future * Who will review new features (Both Redhat and Ericsson will increase review attention for this project) * Fixing the scheduler: ** New scheduler whith which the user can select which host/dragent to schedule ** https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/780675 * OVN and neutron-dynamic-routing ** add the missing features to the OVN gap list ( https://docs.openstack.org/neutron/latest/ovn/gaps.html ) ## RBAC discussion continued with TC * https://etherpad.opendev.org/p/tc-yoga-ptg ## ML2/OVS -> ML2/OVN migration - continue We continued the discussion from Monday, on how to integrate the ovs-ovn migration tool to Neutron CI. ## Team photo: https://photos.app.goo.gl/8ERv59XYoFVQz5oT7 I would like to thank you for the great discussions during last week, it was really useful and productive week. Based on these we have some plan what we will work on in the Yoga cycle. Lajos Katona (lajoskatona) Ericsson Software Technologies

1 0

CentOS-9 guests & 'qemu64' CPU model are incompatible; and reasons to avoid 'qemu64' in general
by Kashyap Chamarthy 26 Oct '21

26 Oct '21

Summary ------- RHEL-9 / CentOS-9 (but not Fedora) has switched[1] to a new baseline microarchitecture called "x86-64-v2". This is to bring in support for additioal low-level CPU instructions, among other reasons. Now, if you've explicitly configured "cpu_mode=none" in `nova.conf` on your compute nodes — which results in the guest getting the extremely undesirable "qemu64" CPU model — it will refuse to boot RHEL-9 or CentOS-9 guests. To fix this, please update the CPU model to "Nehalem". It is the oldest CPU model that is compatible with CentOS-9/RHEL-9 "x86-64-v2". Further, Nehalem also works with `virt_type=kvm|qemu`, _and_ on both Intel and AMD hardware. So this is a good alternative. Details ------- Nova has three config attributes to setup various aspect of a guest CPU: `cpu_mode`, `cpu_model[s]`, and `cpu_model_extra_flags`. Examples of how to use these are in the documentation[2]. If you're using `cpu_mode = none` (e.g. upstream DevStack defalts to it for understandable reasons, mainly live-migration compatiblity): [libvirt] cpu_mode = none ... and want to boot CentOS-9, replace the above with the custom model, "Nehalem", which is the oldest CPU model that's compatible with the new x86-64-v2 baseline: [libvirt] cpu_mode = custom cpu_model = Nehalem The same applies if you're using "qemu64" or "kvm64" with, or without any custom CPU flags — i.e. use Nehalem. (Also, please refer to[3] for more fine-grained recommendations of guest CPU configuration. It's a long document, but a patient reader will be rewarded.) Why is "qemu64" model undesirable for production? ------------------------------------------------- For those wondering about it, a few reasons why `qemu64` CPU model is not at all desirable: (1) It is vulnerable to many of the Spectre and other side-channel security flaws. To see this in "action", you can launch a guest with 'qemu64' CPU model, and then run the below: $ cd /sys/devices/system/cpu/vulnerabilities/ $ grep . * l1tf:Mitigation: PTE Inversion mds:Vulnerable: ... no microcode; SMT Host state unknown meltdown:Mitigation: PTI spec_store_bypass:Vulnerable spectre_v1:Mitigation: usercopy/swapgs barriers ... spectre_v2:Mitigation: Full generic retpoline ... Notice the "Vulnerable" entries. (2) "qemu64" does not support several critical CPU features: (a) AES (Advanced Encryption Standard) instruction, which is important for imporved TLS performance and encryption. (b) RDRAND instruction: without this, guests can get starved for entropy. (c) PCID flag: an obscure-but-important flag that'll lower the performance degradation that you incur from the "Meltdown" security fixes. Probably there are more reasons that I don't know of. An understandable reason why CI systems running in a cloud environment go with 'qemu64' is convenience: with 'qemu64', you can live-migrate a guest regardless of its underlying hardware (whether it's Intel or AMD). That's one main reason why upstream DevStack defaults to it. * * * Overall, the thumb-rule here is to either always explicitly specify a "sane" CPU model, based on the recommendations here[3]. Or to use Nova/libvirt's default ("host-model"). [1] https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-l… [2] https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.cp… [3] https://www.qemu.org/docs/master/system/i386/cpu.html#recommendations-for-k… [4] https://opendev.org/openstack/whitebox-tempest-plugin/src/branch/master/.zu… -- /kashyap

2 2

[cinder] reminder: this week's meeting in video+IRC
by Brian Rosmaita 26 Oct '21

26 Oct '21

Quick reminder that this week's Cinder team meeting on Wednesday 27 October, being the final meeting of the month, will be held in both videoconference and IRC at the regularly scheduled time of 1400 UTC. These are the video meeting rules we've agreed to: * Everyone will keep IRC open during the meeting. * We'll take notes in IRC to leave a record similar to what we have for our regular IRC meetings. * Some people are more comfortable communicating in written English. So at any point, any attendee may request that the discussion of the current topic be conducted entirely in IRC. * The meeting will be recorded. connection info: https://bluejeans.com/3228528973 meeting agenda: https://etherpad.opendev.org/p/cinder-yoga-meetings cheers, brian

1 0

[magnum] [fcos33]
by Yasemin DEMİRAL (BILGEM BTE) 25 Oct '21

25 Oct '21

Hi, How can I dowloand fcos 33? I can't find any link for dowloanding it. Yasemin DEMİRAL [ http://www.tubitak.gov.tr/tr/icerik-sorumluluk-reddi ] Senior Researcher at TUBITAK BILGEM B3LAB Safir Cloud Scrum Master Kimden: "Vikarna Tathe" <vikarnatathe(a)gmail.com> Kime: "Ammad Syed" <syedammad83(a)gmail.com> Kk: "openstack-discuss" <openstack-discuss(a)lists.openstack.org> Gönderilenler: 19 Ekim Salı 2021 16:23:20 Konu: Re: Openstack magnum Hi Ammad, Thanks!!! It worked. On Tue, 19 Oct 2021 at 15:00, Vikarna Tathe < [ mailto:vikarnatathe@gmail.com | vikarnatathe(a)gmail.com ] > wrote: Hi Ammad, Yes, fcos34. Let me try with fcos33. Thanks On Tue, 19 Oct 2021 at 14:52, Ammad Syed < [ mailto:syedammad83@gmail.com | syedammad83(a)gmail.com ] > wrote: BQ_BEGIN Hi, Which fcos image you are using ? It looks like you are using fcos 34. Which is currently not supported. Use fcos 33. On Tue, Oct 19, 2021 at 2:16 PM Vikarna Tathe < [ mailto:vikarnatathe@gmail.com | vikarnatathe(a)gmail.com ] > wrote: BQ_BEGIN Hi All, I was able to login to the instance. I see that kubelet service is in activating state. When I checked the journalctl, found the below. Oct 19 05:18:34 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: Started Kubelet via Hyperkube (System Container). Oct 19 05:18:34 kubernetes-cluster-6cdrblcpckny-master-0 bash[6521]: Error: statfs /sys/fs/cgroup/systemd: no such file or directory Oct 19 05:18:34 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: Main process exited, code=exited, status=125/n/a Oct 19 05:18:34 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: Failed with result 'exit-code'. Oct 19 05:18:44 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 18. Oct 19 05:18:44 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: Stopped Kubelet via Hyperkube (System Container). Executed the below command to fix this issue. mkdir -p /sys/fs/cgroup/systemd Now I am getiing the below error. Has anybody seen this issue. failed to get the kubelet's cgroup: mountpoint for cpu not found. Kubelet system container metrics may be missing. failed to get the container runtime's cgroup: failed to get container name for docker process: mountpoint for cpu not found. failed to run Kubelet: mountpoint for not found On Mon, 18 Oct 2021 at 14:09, Vikarna Tathe < [ mailto:vikarnatathe@gmail.com | vikarnatathe(a)gmail.com ] > wrote: BQ_BEGIN BQ_BEGIN Hi Ammad, Thanks for responding. Yes the instance is getting created, but i am unable to login though i have generated the keypair. There is no default password for this image to login via console. openstack server list +--------------------------------------+--------------------------------------+---------+------------------------------------+----------------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+--------------------------------------+---------+------------------------------------+----------------------+----------+ | cf955a75-8cd2-4f91-a01f-677159b57cb2 | k8s-cluster-01-2nyejxo3hyvb-master-0 | ACTIVE | private1=10.100.0.39, 10.14.20.181 | fedora-coreos-latest | m1.large | ssh -i id_rsa [ mailto:core@10.14.20.181 | core(a)10.14.20.181 ] The authenticity of host '10.14.20.181 (10.14.20.181)' can't be established. ECDSA key fingerprint is SHA256:ykEMpwf79/zTMwcELDSI0f66Sxbri56ovGJ+RRwKXDU. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '10.14.20.181' (ECDSA) to the list of known hosts. [ mailto:core@10.14.20.181 | core(a)10.14.20.181 ] : Permission denied (publickey,gssapi-keyex,gssapi-with-mic). On Mon, 18 Oct 2021 at 14:02, Ammad Syed < [ mailto:syedammad83@gmail.com | syedammad83(a)gmail.com ] > wrote: BQ_BEGIN Hi, Can you check if the master server is deployed as a nova instance ? if yes, then login to the instance and check cloud-init and heat agent logs to see the errors. Ammad On Mon, Oct 18, 2021 at 12:03 PM Vikarna Tathe < [ mailto:vikarnatathe@gmail.com | vikarnatathe(a)gmail.com ] > wrote: BQ_BEGIN Hello All, I am trying to create a kubernetes cluster using magnum. Image: fedora-coreos. The stack gets stucked in CREATE_IN_PROGRESS. See the output below. openstack coe cluster list +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ | 9381cd12-af60-40b1-b1c4-d3dcc93e926e | k8s-cluster-01 | ctrlkey | 2 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ openstack stack resource show k8s-cluster-01-2nyejxo3hyvb kube_masters +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | {'refs_map': None, 'removed_rsrc_list': [], 'attributes': None, 'refs': None} | | creation_time | 2021-10-18T06:44:02Z | | description | | | links | [{'href': ' [ http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu… | http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu… ] ', 'rel': 'self'}, {'href': ' [ http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu… | http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu… ] ', 'rel': 'stack'}, {'href': ' [ http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu… | http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu… ] ', 'rel': 'nested'}] | | logical_resource_id | kube_masters | | physical_resource_id | 3da2083f-0b2c-4b9d-8df5-8468e0de3028 | | required_by | ['kube_cluster_deploy', 'etcd_address_lb_switch', 'api_address_lb_switch', 'kube_cluster_config'] | | resource_name | kube_masters | | resource_status | CREATE_IN_PROGRESS | | resource_status_reason | state changed | | resource_type | OS::Heat::ResourceGroup | | updated_time | 2021-10-18T06:44:02Z | +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Vikarna -- Regards, Syed Ammad Ali BQ_END BQ_END BQ_END BQ_END -- Regards, Syed Ammad Ali BQ_END BQ_END

2 3

[Neutron] Bug Deputy Report October 19 - 23
by Jakub Libosvar 25 Oct '21

25 Oct '21

Hi all, I was the bug deputy for the week starting Oct 19. Here is the report, no critical bugs, it was a quiet week, maybe because of the PTG :) Medium ------ * "network_namespace_exists"can timeout in loaded systems Link: https://bugs.launchpad.net/neutron/+bug/1947974 Assigned to Rodolfo Fix proposed: https://review.opendev.org/c/openstack/neutron/+/814868 * Non HA router - missing iptables rule for redirect metadata queries to haproxy Link: https://bugs.launchpad.net/neutron/+bug/1947993 Assigned to Slawek Fix proposed: https://review.opendev.org/c/openstack/neutron/+/814892 * [OVN] Logical router port must have at least one network Edit Link: https://bugs.launchpad.net/neutron/+bug/1948457 Needs an assignee Low --- * qvo ports are not removed correctly when an instance is deleted immediately after creation Link: https://bugs.launchpad.net/neutron/+bug/1948452 Needs an assignee * [OVN] Mech driver fails to delete DHCP options during subnet deletion Link: https://bugs.launchpad.net/neutron/+bug/1948466 Needs an assignee Kuba

2 1

[Glance] Yoga PTG Summary
by Abhishek Kekane 25 Oct '21

25 Oct '21

Hi All, We had our fourth virtual PTG between 18th October to 22nd October 2020. Thanks to everyone who joined the virtual PTG sessions. Using bluejeans app we had lots of discussion around different topics for glance, glance + cinder and Secure RBAC. I have created etherpad [1] with Notes from the session and which also includes the recordings of each discussion. Here is a short summary of the discussions. Tuesday, October 19th # Xena Retrospective On the positive note, we merged a number of useful features this cycle. We managed to implement a project scope of secure RBAC for metadef APIs, Implemented quotas using unified limits and moved policy enforcing closer to API layer. We also manage to wipe out many bugs from our bug backlog. On the other side we need to improve our documentation and API reference guide. Recording: https://bluejeans.com/s/QuCBmIrMuVv - Chapter 1 # Cache API During Xena cycle we managed to start this work and implement the core functionality but failed to merge it due to lack of negative tests and tempest coverage. In the Yoga cycle we are going to focus on adding tempest coverage for the same along with a new API to cache the given image immediately rather than waiting for a periodic job to pre-cache it for us. Recordings: https://bluejeans.com/s/QuCBmIrMuVv - Chapter 2 # Native Image Encryption Unfortunately this topic is sitting in our agenda for the last couple of PTGs. Current update is the core feature is depending on Barbican microversion work and once it is complete then the Consumer API can be functional again. At the moment in Glance we have decided to go ahead and implement the glance side part and instead of having placeholder (barbican consumer API secret register and deletion) code as commented we can have a WIP patch for the same with depending on glance side work. Recordings: https://bluejeans.com/s/QuCBmIrMuVv - Chapter 3 # Default Glance to configure multiple stores Glance has deprecated single stores configuration since Stein cycle and now will start putting efforts to deploy glance using multistore by-default and then remove single store support from glance. This might be going to take a couple of cycles, so in yoga we are going to migrate internal unit and functional tests to use multistore config and also going to modify devstack to deploy glance using multistore configuration for swift and Ceph (for file and cinder it's already supported). Recordings: https://bluejeans.com/s/QuCBmIrMuVv - Chapter 4 # Quotas Usage API In Xena we have implemented quotas for images API using unified limits. This cycle we will add new APIs which will help enduser to get the clear picture of quotas like What is total quota, used quota and remaining quota. So first we are coming up with the spec for the design and then implementation for the same. Recordings: https://bluejeans.com/s/QuCBmIrMuVv - Chapter 5 Wednesday, October 20th 2021 # Policy Refactoring - Part 2 In Xena we have managed to move all policy checks to the API layer. This cycle we need to work on removing dead code of policy and authorization layer. Before removing both the layers we need to make sure that property protection is working as expected and for the same we need to add one job with a POST script to verify that removing auth and policy layer will not break property protection. Recordings: https://bluejeans.com/s/AMZzGObPhK4 - Chapter 1 # Database read only checks We have added new RBAC policy checks which are equivalent to readonly checks in our db layer, e.g. image ownership check, visibility check etc. To kick start work for this dansmith (thanks for volunteering) will work on PoC and abhishekk will work on specs about how we will modify/improve our db layer. # Secure RBAC - System Scope/Project Admin scope In Xena we have managed to move all policy checks to API layer and implemented project scope of metadef APIs. So as of now we have project scope for all glance APIs. During this discussion Security Team has updated us that discussions are still going about how the system scope should be used/implemented and they are planning to introduce a new role 'manager' which will act between 'admin' and 'member' roles. We need to keep an eye on this new development. https://etherpad.opendev.org/p/tc-yoga-ptg - line #446 https://etherpad.opendev.org/p/policy-popup-yoga-ptg - line #122 Recordings: https://bluejeans.com/s/AMZzGObPhK4 - Chapter 3 Thursday, October 21st 2021 # Glance- Interop interlock Due to confusion between timings this discussion didn't happen as planned. The InterOP team has added some questions later to PTG etherpad (refer line no #270). @Glance team please respond to those questions as I will be out for the next couple of weeks. # Upload volume to image in RBD backend In case when we upload a volume as an image to glance's rbd backend, it starts with a 0 size rbd image and performs resize in chunks of 8 MB which makes the operation very slow. Current idea is to pass volume size as image size to avoid these resize operations. As this change will lead to use locations API which are not advisable to use in glance due to security concerns and also checksum and multi-hash will not be available, cinder side will have new config option (default False) to use this optimization if set to True. Also this change needs a glance side to expose ceph pool information, so Rajat (whoami-rajat) will coordinate with the glance team to set up a spec and implement the same. # Upload volume to image when glance backend is cinder Similar to above topic this will have same security concerns and same config option can be used to have this optimization available. Rajat will coordinate with the glance team to implement glance side changes. Note, as we have joined the cinder team, Brian Rosmaita/Rajat will update/share the recording links for above two sessions in glance PTG etherpad once it is available. # Adding multiple tags overrides existing tags We are going to modify the create multiple tags API which will have one boolean parameter (Default to False) in the header to maintain backward compatibility. If it is True then we are going to add new tags to the existing list of tags rather than replacing those. Recordings: https://bluejeans.com/s/@ttsNs8vIFq - Chapter 1 # Delete newly created metadef resource types from DB after deassociating To maintain the consistency with other metadef APIs, we are going to add two new APIs. 1. Create resource types 2. Delete given resource type Recordings: https://bluejeans.com/s/@ttsNs8vIFq - Chapter 2 You will find the detailed information about the same in the PTG etherpad [1] along with the recordings of the sessions. Kindly let me know if you have any questions about the same. [1] https://etherpad.opendev.org/p/yoga-glance-ptg Thank you, Abhishek

2 1