March 2022 - openstack-discuss - lists.openstack.org

[TripleO] Upgrade overcloud Nodes with Puppet Pacemaker Issue on Controller-0
by Anirudh Gupta 11 Mar '22

11 Mar '22

Hi Team I have deployed TripleO Train Undercloud and Overcloud in HA Mode. After which I Upgraded TripleO Undercloud to Ussuri Release. While upgrading the Overcloud Nodes, there were some minor hiccups initially, but after enabling internet and enabling Centos repositories, we resolved all the issues. After executing openstack overcloud upgrade run command for only the controllers, we get some puppet issues for pacemaker service for Controller-0 (Active DC) The same check passed for Controller-1 and Controller-2. All the overcloud nodes have been spawned with the same image, so there must not be any issue. Attaching the logs below: 2022-03-09 15:22:02.510128 | 525400f8-4aa7-5981-5af0-000000006a82 | TASK | Gather podman infos 2022-03-09 15:22:02.869577 | 525400f8-4aa7-5981-5af0-000000006a0f | CHANGED | Enable and start netns-placeholder | overcloud-controller-1 2022-03-09 15:22:02.873413 | 525400f8-4aa7-5981-5af0-000000006a0f | TIMING | tripleo_container_manage : Enable and start netns-placeholder | overcloud-controller-1 | 0:14:39.816795 | 0.97s 2022-03-09 15:22:02.901485 | 525400f8-4aa7-5981-5af0-000000006940 | TASK | Update container configs with new config hashes 2022-03-09 15:22:02.943620 | 525400f8-4aa7-5981-5af0-000000006940 | TIMING | tripleo_container_manage : Update container configs with new config hashes | overcloud-controller-1 | 0:14:39.887003 | 0.04s 2022-03-09 15:22:02.960739 | 1fecaf23-8a62-488a-87ab-a2ee3c307ae5 | INCLUDED | /usr/share/ansible/roles/tripleo_container_manage/tasks/puppet_config.yml | overcloud-controller-1 2022-03-09 15:22:02.988374 | 525400f8-4aa7-5981-5af0-000000006ab5 | TASK | Update config hashes for container startup configs 2022-03-09 15:22:03.380251 | 525400f8-4aa7-5981-5af0-000000006ab5 | OK | Update config hashes for container startup configs | overcloud-controller-1 2022-03-09 15:22:03.381768 | 525400f8-4aa7-5981-5af0-000000006ab5 | TIMING | tripleo_container_manage : Update config hashes for container startup configs | overcloud-controller-1 | 0:14:40.325159 | 0.39s 2022-03-09 15:22:03.409190 | 525400f8-4aa7-5981-5af0-000000006941 | TASK | Delete orphan containers from /var/lib/tripleo-config/container-startup-config/step_3 2022-03-09 15:22:03.453834 | 525400f8-4aa7-5981-5af0-000000006941 | TIMING | tripleo_container_manage : Delete orphan containers from /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-1 | 0:14:40.397197 | 0.04s 2022-03-09 15:22:03.470413 | 8e8040a9-b1ed-498c-becb-4a25381f487f | INCLUDED | /usr/share/ansible/roles/tripleo_container_manage/tasks/delete_orphan.yml | overcloud-controller-1 2022-03-09 15:22:03.498978 | 525400f8-4aa7-5981-5af0-000000006ad8 | TASK | Gather podman infos instead.\nTraceback (most recent call last):\n File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1039, in resource_update\n resource.getAttribute(\"type\"),\n File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/facade.py\", line 200, in facade_from_parsed_name\n parse_metadata(name, load_metadata(self._runner, name))\n File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 114, in load_metadata\n return _metadata_xml_to_dom(_load_metadata_xml(runner, agent_name))\n File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 52, in _load_metadata_xml\n raise UnableToGetAgentMetadata(agent_name.full_name, stderr.strip())\npcs.lib.resource_agent.error.UnableToGetAgentMetadata\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/sbin/pcs\", line 11, in <module>\n load_entry_point('pcs===0.10.11.22-9d83b', 'console_scripts', 'pcs')()\n File \"/usr/lib/python3.6/site-packages/pcs/app.py\", line 278, in main\n utils.get_library_wrapper(), argv, utils.get_input_modifiers()\n File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router\n return cmd_map[sub_cmd](lib, argv_next, modifiers)\n File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router\n return cmd_map[sub_cmd](lib, argv_next, modifiers)\n File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1058, in resource_update\n [lib_ra.resource_agent_error_to_report_item(e, severity)]\n File \"/usr/lib/python3.6/site-packages/pcs/cli/reports/output.py\", line 38, in process_library_reports\n report_dto = report_item.to_dto()\n File \"/usr/lib/python3.6/site-packages/pcs/common/reports/item.py\", line 182, in to_dto\n severity=self.severity.to_dto(),\nAttributeError: 'str' object has no attribute 'to_dto'\nError: /Stage[main]/Tripleo::Profile::Pacemaker::Ovn_dbs_bundle/Pacemaker::Resource::Ocf[ovndb_servers]/Pcmk_resource[ovndb_servers]: Could not evaluate: pcmk_resource_has_changed? --force resource update ovndb_servers ocf:ovn:ovndb-servers master_ip=[fd00:fd00:fd00:2000::53] nb_master_port=6641 sb_master_port=6642 manage_northd=yes inactive_probe_interval=180000 listen_on_master_ip_only=yes meta notify=true container-attribute-target=host op start timeout=200s stop timeout=200s monitor interval=10s role=Master timeout=60s monitor interval=30s role=Slave timeout=60s bundle ovn-dbs-bundle returned error ovndb_servers. This should never happen.\nWarning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.", "stderr_lines": ["Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5", " (file: /etc/puppet/hiera.yaml)", "Warning: Undefined variable '::deploy_config_name'; ", " (file & line not available)", "Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/6.14/deprecated_language.html", " (file & line not available)", "*Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Ipv6 instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-e… <https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-e…>. at [\"/etc/puppet/modules/tripleo/manifests/profile/pacemaker/ovn_dbs_bundle.pp\", 336]:[\"unknown\", 1]", "* (location: /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:34:in `deprecation')", "Warning: This command is deprecated and will be removed. Please use 'pcs property config' instead.", "Warning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.", "Warning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.", "Traceback (most recent call last):", " File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1039, in resource_update", " resource.getAttribute(\"type\"),", " File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/facade.py\", line 200, in facade_from_parsed_name", " parse_metadata(name, load_metadata(self._runner, name))", " File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 114, in load_metadata", " return _metadata_xml_to_dom(_load_metadata_xml(runner, agent_name))", " File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 52, in _load_metadata_xml", " raise UnableToGetAgentMetadata(agent_name.full_name, stderr.strip())", "pcs.lib.resource_agent.error.UnableToGetAgentMetadata", "", "During handling of the above exception, another exception occurred:", "", "Traceback (most recent call last):", " File \"/sbin/pcs\", line 11, in <module>", " load_entry_point('pcs===0.10.11.22-9d83b', 'console_scripts', 'pcs')()", " File \"/usr/lib/python3.6/site-packages/pcs/app.py\", line 278, in main", " utils.get_library_wrapper(), argv, utils.get_input_modifiers()", " File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router", " return cmd_map[sub_cmd](lib, argv_next, modifiers)", " File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router", " return cmd_map[sub_cmd](lib, argv_next, modifiers)", " File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1058, in resource_update", " [lib_ra.resource_agent_error_to_report_item(e, severity)]", " File \"/usr/lib/python3.6/site-packages/pcs/cli/reports/output.py\", line 38, in process_library_reports", " report_dto = report_item.to_dto()", " File \"/usr/lib/python3.6/site-packages/pcs/common/reports/item.py\", line 182, in to_dto", " severity=self.severity.to_dto(),", "AttributeError: 'str' object has no attribute 'to_dto'", "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Ovn_dbs_bundle/Pacemaker::Resource::Ocf[ovndb_servers]/Pcmk_resource[ovndb_servers]: Could not evaluate: pcmk_resource_has_changed? --force resource update ovndb_servers ocf:ovn:ovndb-servers master_ip=[fd00:fd00:fd00:2000::53] nb_master_port=6641 sb_master_port=6642 manage_northd=yes inactive_probe_interval=180000 listen_on_master_ip_only=yes meta notify=true container-attribute-target=host op start timeout=200s stop timeout=200s monitor interval=10s role=Master timeout=60s monitor interval=30s role=Slave timeout=60s bundle ovn-dbs-bundle returned error ovndb_servers. This should never happen.", "Warning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead."], "stdout": "Notice: Compiled catalog for overcloud-controller-0.localdomain in environment production in 0.75 seconds\nNotice: Applied catalog in 30.15 seconds\nChanges:\nEvents:\n Failure: 1\n Total: 1\nResources:\n Failed: 1\n Out of sync: 1\n Skipped: 26\n Total: 35\nTime:\n Config retrieval: 0.84\n Pcmk resource: 11.39\n Last run: 1646819523\n Pcmk constraint: 2.86\n Transaction evaluation: 30.09\n Catalog application: 30.15\n Pcmk bundle: 7.09\n Pcmk property: 8.43\n Total: 30.15\nVersion:\n Config: 1646819492\n Puppet: 6.14.0", "stdout_lines": ["Notice: Compiled catalog for overcloud-controller-0.localdomain in environment production in 0.75 seconds", "Notice: Applied catalog in 30.15 seconds", "Changes:", "Events:", " Failure: 1", " Total: 1", "Resources:", " Failed: 1", " Out of sync: 1", " Skipped: 26", " Total: 35", "Time:", " Config retrieval: 0.84", " Pcmk resource: 11.39", " Last run: 1646819523", " Pcmk constraint: 2.86", " Transaction evaluation: 30.09", " Catalog application: 30.15", " Pcmk bundle: 7.09", " Pcmk property: 8.43", " Total: 30.15", "Version:", " Config: 1646819492", " Puppet: 6.14.0"]} 2022-03-09 15:22:03.686083 | 525400f8-4aa7-5981-5af0-0000000067ae | FATAL | Run init bundle puppet on the host for ovn_dbs | overcloud-controller-0 | error={"changed": false, "cmd": "puppet apply --detailed-exitcodes --summarize --color=false --modulepath '/etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules' --tags 'pacemaker::resource::bundle,pacemaker::property,pacemaker::resource::ip,pacemaker::resource::ocf,pacemaker::constraint::order,pacemaker::constraint::colocation' -e 'include ::tripleo::profile::base::pacemaker; include ::tripleo::profile::pacemaker::ovn_dbs_bundle'\n", "delta": "0:00:36.507726", "end": "2022-03-09 15:22:03.635619", "failed_when_result": true, "msg": "non-zero return code", "rc": 4, "start": "2022-03-09 15:21:27.127893", "stderr": "Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5\n (file: /etc/puppet/hiera.yaml)\nWarning: Undefined variable '::deploy_config_name'; \n (file & line not available)\nWarning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/6.14/deprecated_language.html\n (file & line not available)\nWarning: This method is deprecated, please use match expressions with Stdlib::Compat::Ipv6 instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-e…. at [\"/etc/puppet/modules/tripleo/manifests/profile/pacemaker/ovn_dbs_bundle.pp\", 336]:[\"unknown\", 1]\n (location: /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:34:in `deprecation')\nWarning: This command is deprecated and will be removed. Please use 'pcs property config' instead.\nWarning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.\nWarning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.\nTraceback (most recent call last):\n File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1039, in resource_update\n resource.getAttribute(\"type\"),\n File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/facade.py\", line 200, in facade_from_parsed_name\n parse_metadata(name, load_metadata(self._runner, name))\n File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 114, in load_metadata\n return _metadata_xml_to_dom(_load_metadata_xml(runner, agent_name))\n File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 52, in _load_metadata_xml\n raise UnableToGetAgentMetadata(agent_name.full_name, stderr.strip())\npcs.lib.resource_agent.error.UnableToGetAgentMetadata\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/sbin/pcs\", line 11, in <module>\n load_entry_point('pcs===0.10.11.22-9d83b', 'console_scripts', 'pcs')()\n File \"/usr/lib/python3.6/site-packages/pcs/app.py\", line 278, in main\n utils.get_library_wrapper(), argv, utils.get_input_modifiers()\n File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router\n return cmd_map[sub_cmd](lib, argv_next, modifiers)\n File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router\n return cmd_map[sub_cmd](lib, argv_next, modifiers)\n File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1058, in resource_update\n [lib_ra.resource_agent_error_to_report_item(e, severity)]\n File \"/usr/lib/python3.6/site-packages/pcs/cli/reports/output.py\", line 38, in process_library_reports\n report_dto = report_item.to_dto()\n File \"/usr/lib/python3.6/site-packages/pcs/common/reports/item.py\", line 182, in to_dto\n severity=self.severity.to_dto(),\nAttributeError: 'str' object has no attribute 'to_dto'\nError: /Stage[main]/Tripleo::Profile::Pacemaker::Ovn_dbs_bundle/Pacemaker::Resource::Ocf[ovndb_servers]/Pcmk_resource[ovndb_servers]: Could not evaluate: pcmk_resource_has_changed? --force resource update ovndb_servers ocf:ovn:ovndb-servers master_ip=[fd00:fd00:fd00:2000::53] nb_master_port=6641 sb_master_port=6642 manage_northd=yes inactive_probe_interval=180000 listen_on_master_ip_only=yes meta notify=true container-attribute-target=host op start timeout=200s stop timeout=200s monitor interval=10s role=Master timeout=60s monitor interval=30s role=Slave timeout=60s bundle ovn-dbs-bundle returned error ovndb_servers. This should never happen.\nWarning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.", "stderr_lines": ["Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5", " (file: /etc/puppet/hiera.yaml)", "Warning: Undefined variable '::deploy_config_name'; ", " (file & line not available)", "Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/6.14/deprecated_language.html", " (file & line not available)", "Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Ipv6 instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-e…. at [\"/etc/puppet/modules/tripleo/manifests/profile/pacemaker/ovn_dbs_bundle.pp\", 336]:[\"unknown\", 1]", " (location: /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:34:in `deprecation')", "Warning: This command is deprecated and will be removed. Please use 'pcs property config' instead.", "Warning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.", "Warning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead.", "Traceback (most recent call last):", " File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1039, in resource_update", " resource.getAttribute(\"type\"),", " File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/facade.py\", line 200, in facade_from_parsed_name", " parse_metadata(name, load_metadata(self._runner, name))", " File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 114, in load_metadata", " return _metadata_xml_to_dom(_load_metadata_xml(runner, agent_name))", " File \"/usr/lib/python3.6/site-packages/pcs/lib/resource_agent/xml.py\", line 52, in _load_metadata_xml", " raise UnableToGetAgentMetadata(agent_name.full_name, stderr.strip())", "pcs.lib.resource_agent.error.UnableToGetAgentMetadata", "", "During handling of the above exception, another exception occurred:", "", "Traceback (most recent call last):", " File \"/sbin/pcs\", line 11, in <module>", " load_entry_point('pcs===0.10.11.22-9d83b', 'console_scripts', 'pcs')()", " File \"/usr/lib/python3.6/site-packages/pcs/app.py\", line 278, in main", " utils.get_library_wrapper(), argv, utils.get_input_modifiers()", " File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router", " return cmd_map[sub_cmd](lib, argv_next, modifiers)", " File \"/usr/lib/python3.6/site-packages/pcs/cli/common/routing.py\", line 33, in _router", " return cmd_map[sub_cmd](lib, argv_next, modifiers)", " File \"/usr/lib/python3.6/site-packages/pcs/resource.py\", line 1058, in resource_update", " [lib_ra.resource_agent_error_to_report_item(e, severity)]", " File \"/usr/lib/python3.6/site-packages/pcs/cli/reports/output.py\", line 38, in process_library_reports", " report_dto = report_item.to_dto()", " File \"/usr/lib/python3.6/site-packages/pcs/common/reports/item.py\", line 182, in to_dto", " severity=self.severity.to_dto(),", "AttributeError: 'str' object has no attribute 'to_dto'", "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Ovn_dbs_bundle/Pacemaker::Resource::Ocf[ovndb_servers]/Pcmk_resource[ovndb_servers]: Could not evaluate: pcmk_resource_has_changed? --force resource update ovndb_servers ocf:ovn:ovndb-servers master_ip=[fd00:fd00:fd00:2000::53] nb_master_port=6641 sb_master_port=6642 manage_northd=yes inactive_probe_interval=180000 listen_on_master_ip_only=yes meta notify=true container-attribute-target=host op start timeout=200s stop timeout=200s monitor interval=10s role=Master timeout=60s monitor interval=30s role=Slave timeout=60s bundle ovn-dbs-bundle returned error ovndb_servers. This should never happen.", *"Warning: This command is deprecated and will be removed. Please use 'pcs constraint config' instead."], "stdout":* "Notice: Compiled catalog for overcloud-controller-0.localdomain in environment production in 0.75 seconds\nNotice: Applied catalog in 30.15 seconds\nChanges:\nEvents:\n Failure: 1\n Total: 1\nResources:\n Failed: 1\n Out of sync: 1\n Skipped: 26\n Total: 35\nTime:\n Config retrieval: 0.84\n Pcmk resource: 11.39\n Last run: 1646819523\n Pcmk constraint: 2.86\n Transaction evaluation: 30.09\n Catalog application: 30.15\n Pcmk bundle: 7.09\n Pcmk property: 8.43\n Total: 30.15\nVersion:\n Config: 1646819492\n Puppet: 6.14.0", "stdout_lines": ["Notice: Compiled catalog for overcloud-controller-0.localdomain in environment production in 0.75 seconds", "Notice: Applied catalog in 30.15 seconds", "Changes:", "Events:", " Failure: 1", " Total: 1", "Resources:", " Failed: 1", " Out of sync: 1", " Skipped: 26", " Total: 35", "Time:", " Config retrieval: 0.84", " Pcmk resource: 11.39", " Last run: 1646819523", " Pcmk constraint: 2.86", " Transaction evaluation: 30.09", " Catalog application: 30.15", " Pcmk bundle: 7.09", " Pcmk property: 8.43", " Total: 30.15", "Version:", " Config: 1646819492", " Puppet: 6.14.0"]} 2022-03-09 15:22:03.690015 | 525400f8-4aa7-5981-5af0-0000000067ae | TIMING | tripleo_ha_wrapper : Run init bundle puppet on the host for ovn_dbs | overcloud-controller-0 | 0:14:40.633394 | 36.87s 2022-03-09 15:22:05.269139 | 525400f8-4aa7-5981-5af0-000000006a82 | OK | Gather podman infos | overcloud-controller-2 2022-03-09 15:22:05.270057 | 525400f8-4aa7-5981-5af0-000000006a82 | TIMING | tripleo_container_manage : Gather podman infos | overcloud-controller-2 | 0:14:42.213453 | 2.76s 2022-03-09 15:22:05.726794 | 525400f8-4aa7-5981-5af0-000000006a83 | TASK | Delete orphan containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 2022-03-09 15:22:05.786601 | 525400f8-4aa7-5981-5af0-000000006a83 | TIMING | Delete orphan containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-2 | 0:14:42.729983 | 0.06s 2022-03-09 15:22:06.038548 | 525400f8-4aa7-5981-5af0-000000006b27 | TIMING | tripleo_container_rm : include_tasks | overcloud-controller-2 | 0:14:42.981908 | 0.19s 2022-03-09 15:22:06.068012 | 525400f8-4aa7-5981-5af0-000000006942 | TASK | Create containers from /var/lib/tripleo-config/container-startup-config/step_3 2022-03-09 15:22:06.119081 | 525400f8-4aa7-5981-5af0-000000006942 | TIMING | tripleo_container_manage : Create containers from /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-2 | 0:14:43.062462 | 0.05s 2022-03-09 15:22:06.136762 | f2c26285-bf89-4ee3-a43f-cdf6e8722aaa | INCLUDED | /usr/share/ansible/roles/tripleo_container_manage/tasks/create.yml | overcloud-controller-2 Regards Anirudh Gupta

1 0

[designate] Architecture help for designate on multi-region/multi-AZ deployments
by Jean-Philippe Evrard 11 Mar '22

11 Mar '22

Hello folks, I currently have designate installed in a single region in a cloud. Now we are thinking about the next steps: A multi-region designate for a simple use case: Bring your own domain, have A/AAAA records for your instances regardless of the region. Side note: That requirement can be modified if what I am asking is impossible. I didn't find anything in the ML, nor in the bug tracker [1] outside a neutron bug [2] (am I blind?). I had a look at the designate documentation, but there are no best practices for multi-region (or am I blind again?) The only relevant part for integration with neutron is in [3], which is not region related. I am looking for advice from designate experts here, to see what we can do. Right now, we are thinking about two ways to deal with this: A) Install a separate openstack with designate, and make all the regions's neutrons configuration point to a single designate. Neutron would work, designate would be simple to manage (as we have a single designate). The downside here is the clear-cut separation of regions: I am afraid of how records are stored in the central designate, and if there won't be multi-tenancy issues (uuid could technically overlap, as different regions means different keystones here). B) Install a designate in each region, with DNS servers in each region. We have strict region separation this way. The downside is the user friendliness: A customer _can_ decide which region in which the domain will be hosted. That means that we now need to worry about synchronizing region's state (if you have created a domain in that region, you can't do it elsewhere). Am I misunderstanding/not seeing something obvious here? Thank you in advance. Jean-Philippe Evrard (evrardjp) [0]: https://www.google.com/search?q=designate+multi+region+site%3Ahttp%3A%2F%2F… [1]: https://bugs.launchpad.net/designate?field.searchtext=region [2]: https://review.opendev.org/c/openstack/neutron/+/684854/ [3]: https://docs.openstack.org/neutron/latest/admin/config-dns-int-ext-serv.html

2 2

IPv6 deployment on OpenStack
by Marc-Antoine Godde 11 Mar '22

11 Mar '22

Hello. We are progressively adding support for IPv6 in my company. We decided to use SLAAC only for laptops, phones, … since DHCPv6 isn’t supported on Android. RDNSS support will also increase. We are now planning our deployment on OpenStack. We already know that we'll rely only on neutron but we are not yet fixed between DHCPv6 and SLAAC ? Do you have any arguments for one these for VMs ? Thanks, Marc-Antoine.

4 11

[neutron] Drivers meeting - Friday 11.3.2022 - cancelled
by Lajos Katona 10 Mar '22

10 Mar '22

Hi Neutron Drivers! Due to the lack of agenda, let's cancel tomorrow's drivers meeting. See You on the meeting next week. Lajos Katona (lajoskatona)

1 0

Re: [trove][tc][all] Call for volunteers to be a PTL and maintainers
by Slawek Kaplonski 10 Mar '22

10 Mar '22

Hi, Once again thx for stepping up for this role in Trove. Patch for that is already proposed https://review.opendev.org/c/openstack/governance/+/833152[1] - we need now Your +1 on this patch to be able to approve it and to make You officially PTL of Trove. On czwartek, 10 marca 2022 12:27:43 CET W Ch wrote: > thanks > > Slawek Kaplonski <skaplons(a)redhat.com> 于2022年3月10日周四 19:06写道： > > > Hi, > > > > On czwartek, 10 marca 2022 11:46:11 CET W Ch wrote: > > > hi, Is anyone elected PTL for trove now ? If nobody, I'd like to > > maintain > > > the trove project, since we are using it in our production. > > > > Thx a lot for stepping up and willing to help with Trove. There are no > > other candidates yet AFAICT so we will discuss Your candidacy and will let > > You know about it soon. > > > > > > > > > > > > > > wu.chunyang > > > > > > Slawek Kaplonski <skaplons(a)redhat.com> 于2022年3月6日周日 01:33写道： > > > > > > > Hi, > > > > > > > > After last PTL elections [1] Trove project still don't have any PTL > > > > elected. > > > > So this is call for maintainers for Trove. If You are using it or > > > > interested in it, and if You are willing to help maintaining this > > project, > > > > please contact TC members through this mailing list or directly on the > > > > #openstack-tc channel @OFTC. We can talk possibilities to make someone > > a > > > > PTL of the project or going with this project to the Distributed > > Project > > > > Leadership [2] model. > > > > > > > > [1] > > > > > > http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027411… > > > > [2] > > > > > > https://governance.openstack.org/tc/resolutions/20200803-distributed-projec… > > > > > > > > -- > > > > Slawek Kaplonski > > > > Principal Software Engineer > > > > Red Hat > > > > > > > > > > > -- > > Slawek Kaplonski > > Principal Software Engineer > > Red Hat > -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://review.opendev.org/c/openstack/governance/+/833152

1 0

[nova] VM ending up in a different availability zone after resizing vm instance
by hai wu 10 Mar '22

10 Mar '22

VM ends up in a different availability zone after resizing the vm instance. The default openstack nova behavior should have prevented this from happening, correct? What could have caused this?

3 4

[oslo] [tooz] Deprecating etcd3 driver
by Tobias Urdin 10 Mar '22

10 Mar '22

Hello, I’m proposing that we deprecate the etcd3 driver https://review.opendev.org/c/openstack/tooz/+/833107 I will include the commit message below that describes the situation with the dependency the driver has on the python-etcd3 library. Please let me know what you think about this action, we would also like to know if you are using the driver. Best regards Tobias ——— Author: Tobias Urdin <tobias.urdin(a)binero.se<mailto:tobias.urdin@binero.se>> Date: Thu Mar 10 15:09:58 2022 +0000 Deprecate the etcd3 driver The etcd3 driver depends on the python-etcd3 [1] library that is currently stale and no has no active maintenance [2] with a non-working CI that blocks all PRs. It's also currently blocking Tooz from supporting newer versions of tenacity [3] that a lot of projects is now pinning because newer version of tenacity dropped old legacy compatibility code that has been in there for years. We currently have etcd3gw as an alternative driver where the library the driver depends on is a deliverable that we maintain. [1] https://github.com/kragniz/python-etcd3 [2] https://github.com/kragniz/python-etcd3/issues/1719 [3] https://review.opendev.org/c/openstack/tooz/+/829412 Change-Id: Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac

2 1

[TC][PTL][ops][all] Release cadence, deprecation, and testing
by Dan Smith 10 Mar '22

10 Mar '22

Hi all, As you probably already know from gmann's reporting, the TC recently passed a resolution[1] regarding the release cadence. This, in response to some discussion around moving to different release cycle lengths, becomes mostly a change regarding our deprecation procedures, as well as a call to support the ability to skip certain releases (designated "tock" releases) while sticking on a yearly cycle (designated "tick" releases). In support of the goal of moving to this new cadence for the AA release (i.e. the yet unnamed release to come after Zed), I have introduced[2] a new grenade-skip-level job, which tests skipping "tock" releases and going straight from one tick to the following. We intend this to be a voting job in AA, making sure it works from Yoga->AA. Right now it tests Wallaby->Yoga for informational purposes. We would recommend projects enable that as voting, or define their own job to do similar testing as soon as possible. Manila has already started[3] this. Note that existing grenade testing between each adjacent release remains unchanged in this scheme. On the topic of deprecation, we moved the current definition of deprecation procedures from the soon-to-be-removed tag definition to a new document in project-team-guide[4]. This now describes the procedures in terms of the tick-tock release arrangement, and is worth a read. As the PTG is coming up, we will certainly discuss this topic in the PTL Interaction[5] session. It would also be advisable to discuss the project-specific details of this in the various project sessions. By identifying this plan early and setting our goals for AA, we have a good amount of time between now and then to get our heads wrapped around what (if any) things we need to be doing. Luckily, the introduction of the job to test this from wallaby to yoga required almost no project changes to get working, which is a good indicator that it won't be too laborious for most projects to adopt. --Dan 1: https://governance.openstack.org/tc/resolutions/20220210-release-cadence-ad… 2: https://review.opendev.org/c/openstack/grenade/+/826101 3: https://review.opendev.org/c/openstack/manila/+/830277 4: https://docs.openstack.org/project-team-guide/deprecation.html 5: https://etherpad.opendev.org/p/tc-ptl-interaction-zed

3 5

[ops][cinder] reminder: resource count survey - win fabulous prizes!
by Brian Rosmaita 10 Mar '22

10 Mar '22

Hello operators, Don't miss out on this chance to take the Cinder Resource Count Survey! Act now before the survey closes on Wednesday 30 March! https://rosmaita.wufoo.com/forms/cinder-resource-count-survey/ Details below: The Cinder project is working on the cinder quotas system in an effort to improve its accuracy and remove the necessity for operators to go through and clean it up occasionally when it gets out of sync with reality. One way to do this is to use dynamic database counting (instead of storing the usage info in a table where it can become outdated). Whether this will be a good change depends on how many records the database has to process during a transaction, which in turn depends on how many Cinder resources a project has. So we need to get some information from you. Please take this brief survey: https://rosmaita.wufoo.com/forms/cinder-resource-count-survey/ The survey closes on Wednesday 30 March at 1200 UTC, so that we can have data to discuss during the Zed PTG. But please don't procrastinate, it's only 6 questions and the answers are all positive integers. It shouldn't take long to fill out! Thanks! brian

1 0

[openstack][queens][cinder] volume retype errors
by Ignazio Cassano 10 Mar '22

10 Mar '22

Hello, sometimes the volume retype from a netapp nfs storage to another netapp nfs storage does not work. I do not know the reason but I think something is going wrong in nova: /var/log/nova/nova-compute.log:2022-02-02 14:23:39.992 194288 ERROR root [req-ebbf749f-359f-4282-94c5-0913cffe97a3 66adb965bef64eaaab2af93ade87e2ca 68feefdf7c5847388534e9214dc13366 - default default] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5655, in _swap_volume\n mountpoint, resize_to)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1647, in swap_volume\n self._disconnect_volume(context, new_connection_info, instance)\n', ' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n', ' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1644, in swap_volume\n self._swap_volume(guest, disk_dev, conf, resize_to)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1593, in _swap_volume\n raise exception.VolumeRebaseFailed(reason=six.text_type(exc))\n', "VolumeRebaseFailed: Volume rebase failed: Requested operation is not valid: pivot of disk 'vda' requires an active copy job\n"]: VolumeAttachmentNotFound: Volume attachment 2cd820e0-85e8-498d-a62a-800260d0cf31 could not be found Must I try to change some parameters in nova.conf on compute nodes about volume attachment ? Thanks Ignazio

3 7