April 2021 - openstack-discuss - lists.openstack.org

[Watcher] Datasource plugins?
by Jing Geng (BLOOMBERG/ 120 PARK) 26 Apr '21

26 Apr '21

Hi There, Currently Watcher supports many kinds of plugins, including actions, cluster data model collector, goals... In fact, it seems everything has plugins except for datasources (metrics). I wonder if there is any reason that so far Watcher doesn't provide plugins for datasources? Another question is, if I'd like to build plugin infrastructure for datasources, which means changing this DataSourceBase class so it extends loadable.Loadable and abc.ABCMeta, like other base classes for plugins, will this be supported by the community? I hope this idea will be supported, especially when there have already been several projects that tried to make datasources more flexible (Formal Datasource Interface, File based Metric Map, Grafana proxy datasource...). It seems to me that we should just provide plugins for datasources so Watcher can work with any metric system that has been deployed. If I missed something and it isn't a good idea, please also let me know why. Looking forward to contributing to the community! Regards, Jing Geng

1 0

[telemetry][ptg] Summary of the discussed topics
by Matthias Runge 26 Apr '21

26 Apr '21

Hi there, last week, we had two days of PTG sessions for telemetry. There were a couple of topics discussed, and here's the summary. - the CI situation for telemetry is not very satisfying. There are a couple of repositories interacting with each other. telemetry gates have been blocked for over a month on the external dependency gnocchi. - We talked about the use (or not using) storyboard for the case of tracking bugs etc. for telemetry. It is/was not very user friendly to report bugs. - for the future, we'll go back and use launchpad for tracking bugs etc. - We had a collab session with the folks from venus, talked about what they are doing, and if there is a room for collaboration. - we also decided to drop panko from OpenStack. A separate mail will follow. Best, Matthias

1 0

RE: [kolla] VM build fails after Train-Ussuri upgrade
by Braden, Albert 26 Apr '21

26 Apr '21

Can anyone help with this upgrade issue? From: Braden, Albert Sent: Monday, April 19, 2021 8:20 AM To: openstack-discuss(a)lists.openstack.org Subject: [kolla] VM build fails after Train-Ussuri upgrade I upgraded my Train test cluster to Ussuri following these instructions: OpenStack Docs: Operating Kolla<https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html#u…> The upgrade completed successfully with no failures, and the existing VMs are fine, but new VM build fails with rados.Rados.connect\nrados.PermissionDeniedError: Ubuntu Pastebin<https://paste.ubuntu.com/p/MJhzgjYbW8/> I'm running external ceph so I looked at this document: OpenStack Docs: External Ceph<https://docs.openstack.org/kolla-ansible/latest/reference/storage/external-…> It says that I need the following in /etc/kolla/config/glance/ceph.conf: auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx I didn't have that, so I added it and then redeployed, but still can't build VMs. I tried adding the same to all copies of ceph.conf and redeployed again, but that didn't help. Does anything else need to change in my ceph config when upgrading from Train to Ussuri? I see some cryptic talk about ceph in the release notes but it's not obvious what I'm being asked to change: OpenStack Docs: Ussuri Series Release Notes<https://docs.openstack.org/releasenotes/kolla-ansible/ussuri.html#relnotes-…> I read the bug that it refers to: Bug #1904062 "external ceph cinder volume config breaks volumes ..." : Bugs : kolla-ansible (launchpad.net)<https://bugs.launchpad.net/kolla-ansible/+bug/1904062> But I already have "backend_host=rbd:volumes" so I don't think I'm hitting that. Also I read these sections but I don't see anything obvious here that needs to be changed: * For cinder (cinder-volume and cinder-backup), glance-api and manila keyrings behavior has changed and Kolla Ansible deployment will not copy those keys using wildcards (ceph.*), instead will use newly introduced variables. Your environment may render unusable after an upgrade if your keys in /etc/kolla/config do not match default values for introduced variables. * The default behavior for generating the cinder.conf template has changed. An rbd-1 section will be generated when external Ceph functionality is used, i.e. cinder_backend_ceph is set to true. Previously it was only included when Kolla Ansible internal Ceph deployment mechanism was used. * The rbd section of nova.conf for nova-compute is now generated when nova_backend is set to "rbd". Previously it was only generated when both enable_ceph was "yes" and nova_backend was set to "rbd". My ceph keys have the default name and are in the default locations. I have cinder_backend_ceph: "yes". I don't have a nova_backend setting but I have nova_backend_ceph: "yes" I added nova_backend: "rbd" and redeployed and now I get a different error: rados.Rados.connect\nrados.ObjectNotFound Ubuntu Pastebin<https://paste.ubuntu.com/p/2kdKCW7npw/> I apologize for the nonsense below. I have not been able to stop it from being attached to my external emails. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

2 2

[TripleO] Xena PTG session summaries
by Marios Andreou 26 Apr '21

26 Apr '21

Hello folks, I sent out some stats and links on our PTG meetup with http://lists.openstack.org/pipermail/openstack-discuss/2021-April/021999.ht… already, but, as a couple of different people asked me about it, I took the time to write a summary for each session today. Of course you can find all etherpad links and recordings via https://etherpad.opendev.org/p/tripleo-ptg-xena (which seems to be down right now but I have backups if it isn't resolved by tomorrow I can try sharing that content somewhere else). Below is a (very) concise summary of the main points in each session and I hope that is useful to someone (especially since it took far longer than I expected ;)). Please reply here (or ping privately if you prefer) for any glaring omissions or obvious issues that should be revised (I originally intended to put this in an etherpad for easier collaboration but as I wrote above, etherpad.opendev.org seems down right now at least for me). regards, marios MON: * https://etherpad.opendev.org/p/tripleo-ptg-retrospective Retrospective of the Wallaby cycle - there are some community and team level 'headlines' on the main items worked on during this cycle on the etherpad. Some identified ideas for improvement include targeting another older branch for end-of-life likely Queens, improving upstream documentation especially removal of stale content, and creating a tag in Launchpad for teams so we can more easily identify which squad is currently assigned. * Topic: Plan/Swift removal update Presentation link: https://drive.google.com/file/d/1igOW4XuAbU55Tat73DwLqO4UGZu8MiNi/view?usp=… An update of the work completed in the W allay cycle to remove the Swift service and the deployment plan (which is no longer used as part of our deployments) from the undercloud. From wallaby onward by default there is no undercloud Swift. There may be a revision of the spec https://opendev.org/openstack/tripleo-specs/commit/e83d8aba3a950da83a33c23b… as the original plan didn't explicitly consider removal of the deployment plan. * https://etherpad.opendev.org/p/tripleo-ephemeral-heat Update on the ephemeral heat work (i.e. no permanent heat process on the undercloud). There has been very strong progress made in this cycle and there are still some outstanding patches https://review.opendev.org/q/topic:%22ephemeral-heat%22+(status:open) to be merged. Goal is to make this the default in Xena deployments and backport to Wallaby as optional. Besides the main feature, some related planned work includes consolidation of the python-tripleoclient "overcloud deploy" and "tripleo deploy" (eg standalone) commands. Note that this work depends on the tripleo-network-v2 work (next session below). * https://etherpad.opendev.org/p/tripleo-network-v2 Update on the network ports v2 work (moving network port creation out of the heat stack) https://opendev.org/openstack/tripleo-specs/src/branch/master/specs/wallaby… - again good progress on this during Wallaby but there is still some ongoing work there https://review.opendev.org/q/topic:%2522network-data-v2%2522+(status:open) . The goal for Xena is to make this the default (i.e. no node/networking config in deploy-steps-playbook.yaml). One main area of work for X in this topic is integration of the baremetal network config in the overcloud deployment (i.e. allow a single command). * https://etherpad.opendev.org/p/tripleo-ceph-xena Update from the ceph team about the main work items completed in Wallaby including the tripleo-ceph-client and tripleo-ceph in place of ceph-ansible for RBD ( https://specs.openstack.org/openstack/tripleo-specs/specs/wallaby/tripleo-c… and https://specs.openstack.org/openstack/tripleo-specs/specs/wallaby/tripleo-c… ). The main work planned for Xena is to continue trying to achieve feature parity with ceph-ansible - including resolving cephadm blockers, , Ganesha & https://specs.openstack.org/openstack/tripleo-specs/specs/wallaby/tripleo-c… . One major consideration is how to move ceph creation/config outside of the heat stack - some parts such as pools, keyrings and haproxy config will have to remain as part of the tripleo deployment. Note that this work depends on the network ports v2 (previous session above). TUE: * https://etherpad.opendev.org/p/tripleo-xena-whole-disk-images A proposal to move to whole disk images instead of the current overcloud-full.qcow+overcloud-full.initrd+overcloud-full.vmlinuz. There were many compelling arguments made for the proposal including: with the overcloud-full.qcow2 partition image, as of centos 8.4 grub2 no longer supports UEFI boot, there will be much less for ironic-python-agent to do during deployment with a single disk image, there will be just one file to distribute (vs 3), we will no longer need to define and build a separate 'hardened' image (and also remove the related CI jobs). One of the main technical issues that needs to be addressed first is the grow partition for /var which is where we are storing containers and config for deployment. * https://etherpad.opendev.org/p/tripleo-xena-drop-healthchecks Proposal to drop the container health check since are using deployment resources but aren't providing value. There was no push back against this proposal and the details are being discussed in the newly posted spec @ https://review.opendev.org/c/openstack/tripleo-specs/+/787535. * https://etherpad.opendev.org/p/ci-tripleo-repos Proposal to consolidate the various ways and places that tripleo-ci is using to configure the repos in the CI jobs. There is a spec proposed @ https://review.opendev.org/c/openstack/tripleo-specs/+/772442 - some of the work here is split into sub items which are ongoing (tripleo-get-hash there https://review.opendev.org/c/openstack/tripleo-ci/+/784392). The main outstanding blocking item here is to agree on the common data format for the various personas upstream downstream and product that we need to support eg https://github.com/mwhahaha/rhos-bootstrap/blob/main/versions/centos.yaml vs https://review.opendev.org/c/openstack/tripleo-repos/+/785593/1/tripleo_rep… * openstack tempest skiplist https://docs.google.com/presentation/d/1aCiV35IYNhPV7SRmi4_A9vkIjZ89pwfC4Vv… Update on the tempest skiplist effort during Wallaby to consolidate the skipped Tempest tests in a central location with the ability to specify particular jobs and or branches for which specific skips will apply. * https://etherpad.opendev.org/p/tripleo-next One of the main items discussed here was the 'first principles' proposal at https://review.opendev.org/c/openstack/tripleo-specs/+/786980 - these are meant to guide us when discussing changes to our deployment tooling and architecture. The proposal will merge in Xena specs once we've reached consensus on the review. Another topic discussed in this session was an update on exploratory work to replace "heat & ansible" in our deployment tooling with 'something else' - some ongoing work here is at https://github.com/cloudnull/director & https://github.com/mwhahaha/task-core. More info and pointers (also discussed Kube/OCP with an operator to deploy tripleo) on the etherpad. WED: * https://etherpad.opendev.org/p/tripleo-xena-inventory-script This was a proposal to remove the "tripleo-ansible-inventory script" @ https://github.com/openstack/tripleo-common/blob/ccd990b58b6583dda3a0e0f341… and instead generate it from the deployment data (e.g. metalsmith or user data from deployed-server deployments). The consensus reached was that instead of removing it we should instead use it in a better way, for example make sure static inventories are generated and exported to known locations (especially for the ephemeral heat case) and re-used. * https://etherpad.opendev.org/p/vf-ui-output This was an update from the validations squad about the main items worked on during Wallaby (integrated the validation framework into the component CI pipelines, enabled the standalone job in upstream check/gate and increased adoption especially by the upgrades squad). Followed by discussions for planned Xena work, including changes in the UI/CLI (eg jq queries can be handled better and various other UI improvements more on the etherpad). Some of the other topics raised here were to make the validations themselves component aware (run all validations related to a given component) and discussion around the requirement for a molecule test on all validation additions (especially the example of mocking out OpenStack services like keystone); the compromise could be to instead use a standalone job for such cases. * https://etherpad.opendev.org/p/Validation-Framework-Next-Generation In this session the validations squad introduced ideas for the future direction of the validation framework. Some of the main proposals are to remove the validations repos - validations-common and validations-libs out of tripleo governance but still within openstack and establishing a new validations project (discussion but no clear consensus on this point), to re-merge the two repos into one consolidated validations repo and fixup the CLI (see previous session) - more items and other considerations on the etherpad. * https://etherpad.opendev.org/p/tripleo-frr-integration Update on Wallaby progress from the cross-squad team looking at FRR/BGP integration in the tripleo deployment (https://opendev.org/openstack/tripleo-specs/src/branch/master/specs/wallaby…). Some of the main items discussed for Xena work included how we might approximate some part of this feature in upstream CI (high resource requirements - downstream CI has 9 nodes) and backport considerations (no backport to upstream/train). * https://etherpad.opendev.org/p/update-upgrade-consolidation In this session the upgrades squad outlined their proposal for consolidation of the minor update and major upgrade workflows - without any blockers or objections coming out of the discussion. One of the main considerations was around how we can decouple the operating system updates/upgrades from the tripleo container upgrade - one action item is to de-containerize those containers that are tied to the kernel version (ABI) such as libvirt and openvswitch. THU: * https://etherpad.opendev.org/p/policy-popup-xena-ptg In this session the security squad gave an update on progress during Wallaby on the Role Based Access Control (RBAC) - many services have completed implementation (Keystone, Nova, Ironic - more on the etherpad). Then there was a discussion around potential integration points during the tripleo deployment, for example https://review.opendev.org/c/openstack/tripleo-heat-templates/+/781571/7/en… . One of the considerations was around how we can test this in CI (possibly the standalone job is a good fit) as well as the use of multiple clouds.yaml for project specific operations during the deployment (with the root clouds yaml having the system-admin profile). * https://etherpad.opendev.org/p/centos-stream-9-upstream In this session the CI squad lead a discussion around centos9 stream (possibly coming Apr/May) and what we should consider/prepare for with respect to upstream CI. Some of the main changes and discussion items included NetworkManager and firewalld replacing iptables, ansible version (2.11/2.12?/?). Mainly this effort is blocked on the actual 9-stream release and getting the relevant nodepool node. Another main discussion point here was whether we would support both stream-8 and stream-9 on particular branches - consensus here is that wallaby has both 8/9 and for X can have only 9 - but this is all dependent on when 9 becomes available with respect to when Xena is released. * https://etherpad.opendev.org/p/os-migrate This session was an update from the upgrades squad around the os-migrate tool ( https://github.com/os-migrate/os-migrate ) - which aims to 'copy' your openstack deployment and in particular the end-user workloads (i.e. user data, vms etc, but not the controlplane) onto new hardware, as an alternative to the in-place upgrade. More information and slides @ https://docs.google.com/presentation/d/1UYGOI89MBLHLpS89mPp0VK1yvTYtb2BamUL…

1 1

[TripleO] next irc meeting Tuesday Apr 27 @ 1400 UTC in #tripleo
by Marios Andreou 26 Apr '21

26 Apr '21

Reminder that the next TripleO irc meeting is: ** Tuesday 27 April at 1400 UTC in #tripleo ** ** https://wiki.openstack.org/wiki/Meetings/TripleO ** ** https://etherpad.opendev.org/p/tripleo-meeting-items ** Please add anything you want to highlight at https://etherpad.opendev.org/p/tripleo-meeting-items This can be recently completed things, ongoing review requests, blocking issues, or anything else tripleo you want to share. Our last meeting was on Mar 13 - you can find the logs there http://eavesdrop.openstack.org/meetings/tripleo/2021/tripleo.2021-04-13-14.… Hope you can make it on Tuesday, regards, marios

1 0

[Interop]: Interop Working Group Meeting
by Kanevsky, Arkady 26 Apr '21

26 Apr '21

1 0

RE: [Interop] Proposal to move weekly meeting 1 hour earlier
by Kanevsky, Arkady 26 Apr '21

26 Apr '21

Thanks Dmitry. You are correct it is for Interop WG not Ironic. Thanks for catching it. That is for Friday weekly Interop Meeting. Arkady From: Dmitry Tantsur <dtantsur(a)redhat.com> Sent: Friday, April 23, 2021 9:12 AM To: Kanevsky, Arkady Cc: openstack-discuss(a)lists.openstack.org Subject: Re: [Ironic] Proposal to move weekly meeting 1 hour earlier [EXTERNAL EMAIL] I suspect you wanted to use [interop] tag, not [ironic] :) On Fri, Apr 23, 2021 at 4:10 PM Kanevsky, Arkady <Arkady.Kanevsky(a)dell.com<mailto:Arkady.Kanevsky@dell.com>> wrote: Team, On today’s PTG Interop meeting we proposed to move weekly Friday meeting 1 hour earlier to 16:00 UTC. Please, respond if the proposed time will or not will work for you. Thanks, Arkady Kanevsky, Ph.D. SP Chief Technologist & DE Dell Technologies office of CTO Dell Inc. One Dell Way, MS PS2-91 Round Rock, TX 78682, USA Phone: 512 7204955 -- Red Hat GmbH, https://de.redhat.com/ [de.redhat.com]<https://urldefense.com/v3/__https:/de.redhat.com/__;!!LpKI!wCVrafnGLyCBhmrC…> , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

4 4

[TripleO] overcloud node introspect failed
by Vinesh N 26 Apr '21

26 Apr '21

hi, i am facing an issue while introspect the bare metal nodes, *error message* "*4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf | 2021-04-22T01:41:32 | 2021-04-22T01:41:35 | Failed to set boot device to PXE: Failed to set boot device for node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf: Client Error for url: http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/manage… <http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/manage…>, Node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf is locked by host undercloud.localdomain, please retry after the current operation is completed*" *(undercloud) [stack@undercloud ~]$ cat /etc/*release* CentOS Linux release 8.3.2011 *ussuri version* *(undercloud) [stack@undercloud ~]$ openstack image list* /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) +--------------------------------------+------------------------+--------+ | ID | Name | Status | +--------------------------------------+------------------------+--------+ | 8ddcd168-cc18-4ce2-97c5-c3502ac471a4 | overcloud-full | active | | 8d9cfac9-400b-4570-b0b1-baeb175b16c4 | overcloud-full-initrd | active | | c561f1d5-41ae-4599-81ea-de2c1e74eae7 | overcloud-full-vmlinuz | active | +--------------------------------------+------------------------+--------+ *Using the command to introspect the node, it was able to discover the node and I could provision the node boot via pxe, and load the image on the node. I could see the login prompt on the server, after some time of provision shut the node down.* *openstack overcloud node discover --range 10.0.40.5 --credentials admin:XXXX --introspect --provide* /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) Successfully probed node IP 10.0.40.5 Successfully registered node UUID 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) PLAY [Baremetal Introspection for multiple Ironic Nodes] *********************** 2021-04-22 07:04:28.978299 | 002590fe-0d22-76eb-1a70-000000000008 | TASK | Check for required inputs 2021-04-22 07:04:29.002729 | 002590fe-0d22-76eb-1a70-000000000008 | SKIPPED | Check for required inputs | localhost | item=node_uuids 2021-04-22 07:04:29.004468 | 002590fe-0d22-76eb-1a70-000000000008 | TIMING | Check for required inputs | localhost | 0:00:00.069134 | 0.0 .... .... .... 2021-04-22 07:11:43.261714 | 002590fe-0d22-76eb-1a70-000000000016 | TASK | Nodes that failed introspection 2021-04-22 07:11:43.296417 | 002590fe-0d22-76eb-1a70-000000000016 | FATAL | Nodes that failed introspection | localhost | error={ "msg": " 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf" } 2021-04-22 07:11:43.297359 | 002590fe-0d22-76eb-1a70-000000000016 | TIMING | Nodes that failed introspection | localhost | 0:07:14.362025 | 0.03s NO MORE HOSTS LEFT ************************************************************* PLAY RECAP ********************************************************************* localhost : ok=4 changed=1 unreachable=0 failed=1 skipped=5 rescued=0 ignored=0 2021-04-22 07:11:43.301553 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-04-22 07:11:43.302101 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 10 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-04-22 07:11:43.302609 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:07:14.367265 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-04-22 07:11:43.303162 | UUID | Info | Host | Task Name | Run Time 2021-04-22 07:11:43.303740 | 002590fe-0d22-76eb-1a70-000000000014 | SUMMARY | localhost | Start baremetal introspection | 434.03s 2021-04-22 07:11:43.304248 | 002590fe-0d22-76eb-1a70-000000000015 | SUMMARY | localhost | Nodes that passed introspection | 0.04s 2021-04-22 07:11:43.304814 | 002590fe-0d22-76eb-1a70-000000000016 | SUMMARY | localhost | Nodes that failed introspection | 0.03s 2021-04-22 07:11:43.305341 | 002590fe-0d22-76eb-1a70-000000000008 | SUMMARY | localhost | Check for required inputs | 0.03s 2021-04-22 07:11:43.305854 | 002590fe-0d22-76eb-1a70-00000000000a | SUMMARY | localhost | Set node_uuids_intro fact | 0.02s 2021-04-22 07:11:43.306397 | 002590fe-0d22-76eb-1a70-000000000010 | SUMMARY | localhost | Check if validation enabled | 0.02s 2021-04-22 07:11:43.306904 | 002590fe-0d22-76eb-1a70-000000000012 | SUMMARY | localhost | Fail if validations are disabled | 0.02s 2021-04-22 07:11:43.307379 | 002590fe-0d22-76eb-1a70-00000000000e | SUMMARY | localhost | Set concurrency fact | 0.02s 2021-04-22 07:11:43.307913 | 002590fe-0d22-76eb-1a70-00000000000c | SUMMARY | localhost | Notice | 0.02s 2021-04-22 07:11:43.308417 | 002590fe-0d22-76eb-1a70-000000000011 | SUMMARY | localhost | Run Validations | 0.02s 2021-04-22 07:11:43.308926 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-04-22 07:11:43.309423 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2021-04-22 07:11:43.310021 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~ 2021-04-22 07:11:43.310545 | The following node(s) had failures: localhost 2021-04-22 07:11:43.311080 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2 Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_node.py", line 462, in take_action retry_timeout=parsed_args.retry_timeout, File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/baremetal.py", line 193, in introspect "retry_timeout": retry_timeout, File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 728, in run_ansible_playbook raise RuntimeError(err_msg) RuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2 Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2 *(undercloud) [stack@undercloud ~]$ openstack baremetal introspection list* /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version! RequestsDependencyWarning) +--------------------------------------+---------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | UUID | Started at | Finished at | Error | +--------------------------------------+---------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf | 2021-04-22T01:41:32 | 2021-04-22T01:41:35 | Failed to set boot device to PXE: Failed to set boot device for node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf: Client Error for url: http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/manage…, Node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf is locked by host undercloud.localdomain, please retry after the current operation is completed. | | 3d091348-e9c7-4e99-80e3-df72d332d935 | 2021-04-21T12:36:30 | 2021-04-21T12:36:32 | Failed to set boot device to PXE: Failed to set boot device for node 3d091348-e9c7-4e99-80e3-df72d332d935: Client Error for url: http://10.0.1.202:6385/v1/nodes/3d091348-e9c7-4e99-80e3-df72d332d935/manage…, Node 3d091348-e9c7-4e99-80e3-df72d332d935 is locked by host undercloud.localdomain, please retry after the current operation is completed. | +--------------------------------------+---------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

2 2

[neutron] April 2021 - PTG Summary
by Slawek Kaplonski 26 Apr '21

26 Apr '21

Hi, Below is my summary of the Neutron sessions during the PTG last week. Etherpad with agenda and all notes from the sessions is available at https:// etherpad.opendev.org/p/neutron-xena-ptg If You want to discuss some topic in more details, please start new thread for it to keep this one clean. ## Day 1 (Monday) ### Retrospective of the Wallaby cycle Good things: * we accomplished our community goals this cycle, like e.g. rootwrap to privsep migration, * migration to the secure RBAC, * many performance improvements, * CI improvements and the job simplification/reducing, * stable team, and a lot of small patches from all around the world, * OVN feature gap with OVS is shrinking, * team support to implement address groups, including RBAC support. In the list of not so good things we mentioned: * During the previous PTG we were talking about some OVN related guides/sessions but we didn't made any, * networking-midonet - (yet another) stadium project deprecated due to lack of maintainers, but it's better to have them clearly marked as out of stadium, than false expectations - we can bring such projects back when needed anyway, * still unstable CI, mainly tempest and tempest-plugin jobs, also during this release functional tests. As for action items for the _Xena_ cycle, we listed couple of things: * Miguel wants to do the ovn knowledge share internally in his company and later share the material, * we will keep working on the improvements for our CI, list of all opened, CI related bugs can be found at https://tinyurl.com/neutron-gate-failures and we will work on reducing that list in next months. * Rodolfo and Miguel will follow up on Nova's plans to drop eventlet and move to the threading and will investigate if we can try to do something similar in the Neutron, * Akihiro raised good point about ML2/OVN and stadium projects. We should clarify the support plan for OVN in all of our stadium projects. Lajos and Lucas volunteered to work on that. ### Secure RBAC - testing - discussion with QA team As next session in the first day, we attended QA team's session about testing new secure RBAC policies. Summary of that discussion can be found in the etherpad: https://etherpad.opendev.org/p/qa-xena-ptg. QA team wants to switch each project to enforce new defaults and the system scope tokens in the Devstack. We will have to monitor that and check what needs to be fixed on our side to accomplish that. ### Deprecated hybrid plug and the iptables_hybrid firewall driver Bence proposed discussion about potential deprecation of the iptables_hybrid firewall driver as we have now native openvswitch driver too. After discussion we decided that we are not going to deprecate it, at least for now. There are known differences between those two drivers and there is still many usecases where people wants to use iptables_hybrid driver. ### Future of the Linuxbridge backend As the last topic of the first day we discussed future of the Linuxbridge backend and ML2 driver. The same topic was discussed in the http://kaplonski.pl/ blog/shanghai_ptg_summary/ and then, based on the feedback from operators we decided that we are not going to deprecate this backend anytime soon as many people are still using it. But now we are couple of cycle later and things didn't change a lot. We still don't have anyone who would like to maintain it. In most cases it works fine but there are also pretty many bugs reported for that backend, which don't have owners - see https://tinyurl.com/linuxbridge-bugs. But things may change in the future and e.g. removal of the legacy iptables from the main distributions may makes things harder to work. We again had feedback, e.g. from Jon that NASA is using that driver and want's to keep using it. But from the other hand in the core Neutron we don't have anyone interested in maintaining it currently. The outcome of the discussion is that we will again raise that topic and ask operators about: - who is using it and what are the use cases addressed by that driver - maybe we can help with proposing another solution, - who is willing to help with maintenance of that backend. If You are using Linuxbridge backend, please give us such feedback. For Xena status is that we still do our best to keep Linuxbridge backend to be running and fully supported but we will also revisit its status again in the next PTG. ## Day 2 (Tuesday) ### Continuation of "L3 feature improvements" During the second day of the PTG we had very interesting discussions about L3 improvements. We discussed bunch of the RFEs proposed by Bence, Lajos and Manu: * [RFE] Add BFD support for Neutron https://bugs.launchpad.net/neutron/+bug/ 1907089 and the spec: https://review.opendev.org/c/openstack/neutron-specs/+/767337 * [RFE] Allow explicit management of default routes: https://bugs.launchpad.net/ neutron/+bug/1921126) and the spec: https://review.opendev.org/c/openstack/neutron-specs/+/781475 * Allow multiple external gateways on a router: https://bugs.launchpad.net/neutron/ +bug/1905295 and the spec: https://review.opendev.org/c/openstack/neutron-specs/+/779511 * [RFE] Enhancement to Neutron BGPaaS to directly support Neutron Routers & bgp-peering from such routers over internal & external Neutron Networks https:// bugs.launchpad.net/neutron/+bug/1921461 and the spec: https://review.opendev.org/c/openstack/neutron-specs/+/783791 * [RFE] BFD for BGP Dynamic Routing: https://bugs.launchpad.net/neutron/+bug/ 1922716 and the spec: https://review.opendev.org/c/openstack/neutron-specs/+/785349. That one still needs to be discussed and approved during the drivers meeting. General outcome from the discussion is that during the Xena cycle we at least wants to have all those specs merged to have agreement about implementation details. When that will be done, Bence, Lajos and others from Ericsson will work on the implementation of those RFEs. ### Tap-as-a-service (taas) project Next topic which we discussed during Tuesday's session was about Tap-as-a-service: https://opendev.org/x/tap-as-a-service project. Few cycle back we were thinking about including it in the Neutron stadium but finally that

1 0

[keystone] Token
by Adam Tomas 26 Apr '21

26 Apr '21

Hi Which CLI setting sets domain_id field in a token? I tried openstack —os-domain-id SOME_OS_COMMAND, openstack —os-default-domain SOME_OS_COMMAND, openstack —os-default-domain_id SOME_OS_COMMAND but none of them sets this field and policies checking domain_id:%(domain_id) don’t work because of that. Interesting thing is that horizon somehow generates token with domain_id set and everything works with the same policies, I have a problem only with CLI. Can user_domain_id (which is inside of every token is see for particular user) be used instead of domain_id? Example token from CLI: 2021-04-23 12:16:38.090 700 DEBUG keystone.server.flask.request_processing.middleware.auth_context [req-117bc600-490e-46ae-a857-0c8d09dc1dbc 9adbxxxxb02ef 61d4xxxx9c0f - 3a08xxxx82c1 3a08xxxx82c1] RBAC: auth_context: {'token': <TokenModel (audit_id=BLWXSpdbTvqc0YS9WzStjQ, audit_chain_id=['BLWXSpdbTvqc0YS9WzStjQ']) at 0x7f8c390aaca0>, 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9adbxxxx02ef', 'user_domain_id': '3a08xxxx82c1', 'system_scope': None, 'project_id': '61d4xxxx9c0f', 'project_domain_id': '3a08xxxx82c1', 'roles': ['member', 'project_admin', 'reader', 'domain_admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} fill_context /var/lib/kolla/venv/lib/python3.8/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py:478 Example token from Horizon: 2021-04-23 12:48:21.009 704 DEBUG keystone.server.flask.request_processing.middleware.auth_context [req-d6d89d3e-c3c1-48c0-b3ed-b3dcedb54db3 9adbxxxx02ef - 3a08xxxx82c1 3a08xxxx82c1 -] RBAC: auth_context: {'token': <TokenModel (audit_id=ZHltw2esTJyTRnFlgHetog, audit_chain_id=['ZHltw2esTJyTRnFlgHetog', 'iJGq-E9fQKKXdZaZq72MQw']) at 0x7f8c3a1b4460>, 'domain_id': '3a08xxx82c1', 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': ‚xxxx', 'group_ids': [], 'user_id': '9adbxxxx02ef', 'user_domain_id': '3a08xxxx82c1', 'system_scope': None, 'project_id': None, 'project_domain_id': None, 'roles': ['project_admin', 'member', 'reader', 'domain_admin'], 'is_admin_project': False, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} fill_context /var/lib/kolla/venv/lib/python3.8/site-packages/keystone/server/flask/request_processing/middleware/auth_context.py:478 Best regards Adam

1 1