We are tickled pink to announce the release of: ironic 22.0.0: OpenStack Bare Metal Provisioning This release is part of the bobcat release series. The source is available from: https://opendev.org/openstack/ironic Download the package from: https://tarballs.openstack.org/ironic/ Please report issues through: https://storyboard.openstack.org/#!/project/943 For more details, please see below. 22.0.0 ^^^^^^ New Features ************ * Add new conductor conf option: [conductor]poweroff_in_cleanfail (default: False). when True nodes entering clean failed state will be powered off. This option may be unsafe when using Cleaning to perform hardware-transformative actions such as firmware upgrade. * Adds the concept of "parent_node" which allows a "child node", such as an independently managed BMC controlled device deployed within a "parent_node" as part of API version *1.83*. Child nodes are hidden from normal node lists as they are not "general purpose" machines, but have a specific embedded usage. In this model, RBAC rules also apply so if you wish an "owner" or "lessee" to have the child node visible, they must also have the the appropriate "owner" or "lessee" value set matching the parent node. * Adds a "/v1/nodes/?include_children=True" parameter to get a list of all nodes and their children. * Adds a "/v1/nodes/?parent_node=<node_ident>" query parameter to permit retrieval of a list of child nodes assigned to the parent denoted by "<node_ident>". * On shutdown the conductor will wait for at most "[DEFAULT]graceful_shutdown_timeout" seconds for existing lock node reservations to clear. Previously lock reservations were cleared immediately, which in some cases would result in nodes going into a failed state. * The Redfish firmware upgrade interface now supports checksum determination by length, and "sha256" and "sha512" checksums may now be supplied to the step arguments. Upgrade Notes ************* * This upgrade contains an additional field for the "nodes" table, named "parent_node". This update also indexes the "parent_node" database column to prevent performance issues in large deployments. * "[DEFAULT]graceful_shutdown_timeout" defaults to 60s. Systemd "TimeoutStopSec" defaults to 30s. Kubernetes "terminationGracePeriodSeconds" defaults to 90s. It is recommended to align the value of "[DEFAULT]graceful_shutdown_timeout" with the graceful timeout of the process manager of the conductor process. * Fully removes the "cpus" property from the documentation and inspect interface implementations. It was never used internally by Ironic, and is no longer used by Nova. * The defaults for "kernel_append_params" have had the Linux kernel command line parameter "nomodeset" removed from the defaults for the "kernel_append_params" settings. The "nomodeset" option is for troubleshooting and changes the behavior of the graphics interface such that memory can be locked upon graphical updates on physical servers with BMC graphical interfaces, which results in spikes in latency and packet loss whenever graphics updates occur. Operators may add the option to their local configuration, but should be aware that large image transfers or other high IO operations can be impacted. Bug Fixes ********* * [bug 2010613 (https://storyboard.openstack.org/#!/story/2010613)] Fixes issue with SNMP v3 auth protocol and priv protocol set in driver info not being retrieved correctly when a SNMP client is initialized. * Fixes Ironic integration with Cinder because of changes which resulted as part of the recent Security related fix in bug 2004555 (https://launchpad.net/bugs/2004555). The work in Ironic to track this fix was logged in bug 2019892 (https://bugs.launchpad.net/ironic/+bug/2019892). Ironic now sends a service token to Cinder, which allows for access restrictions added as part of the original CVE-2023-2088 fix to be appropriately bypassed. Ironic was not vulnerable, but the restrictions added as a result did impact Ironic's usage. This is because Ironic volume attachments are not on a shared "compute node", but instead mapped to the physical machines and Ironic handles the attachment life- cycle after initial attachment. * Fixes "Invalid cross-device link" in some cases when using "file://" image URLs. * Fixes issues in Ironic's use of SQLAlchemy with SQLite Databases, which is common with users like Metal3, which prevented Ironic from supporting SQLAlchemy 2.0 properly, as autocommit was re-enabled. * Fixes bug of iRMC driver in parse_driver_info where, if FIPS is enabled, SNMP version is always required to be version 3 even though iRMC driver's xxx_interface doesn't use SNMP actually. * Fixes bug in iRMC driver, where *irmc* power_interface sets and updates *irmc_ipmi_succeed* flag which is used by rest of iRMC driver code to deal with iRMC firmware's IPMI incompatibility but *ipmitool* power_interface doesn't set nor update *irmc_ipmi_succeed* flag and rest of iRMC driver code fail to handle iRMC firmware's IPMI incompatibility correctly. * Fixes an issue where an agent token could be inadvertently orphaned if a node is already in the target power state when we attempt to turn the node off. * Fixes scope classification check with the "self_owned_node" policy check where it was limited to check execution with only project scoped, so system scoped users who ticked the policy endpoint would basically get an incorrect error. * Enables boot mode switching during anaconda deploy for "ilo" and "ilo5" hardware types. * Fixes secure boot with anaconda deploy. * Fixes the bug where provisioning a Redfish managed node fails if the BMC doesn't support EthernetInterfaces attribute, even if MAC address information is provided manually. This is done by handling of MissingAttributeError sushy exception in get_mac_addresses() method. This fix is needed to successfully provision machines such as Cisco UCSB and UCSX. * No longer re-calculates checksums for images that are already raw. Previously, it would cause significant delays in deploying raw images. * Fixes an issue where the database upgrade can hang on Python 3.10. This was because open transactions could become orphaned awaiting the Python runtime to clean up their memory references due to the way the overall database query was being intiiated to pre-flight check the upgrade. We have structurally changed the behavior to remedy this case. * Agents deploying on physical servers with default kernel arguments were suspetible to packet loss if a Matrox VGA/Aspeed BMC Graphics interface is present on the machine. The defaults have been changed to remove the use of the "nomodeset" kernel command line parameter which should only be used for troubleshooting as it has been determined that the memory updates can lock all of the kernel memory upon any console graphics update which can negatively impact IO for Networking or Disk interactions. * Fixes an issue where an agent token was being orphaned if a baremetal node timed out during cleaning operations, leading to issues where the node would not be able to establish a new token with Ironic upon future in some cases. We now always wipe the token in this case. Changes in ironic 21.4.0..22.0.0 -------------------------------- bf850cad1 Make metal3 job voting d66530494 Add DB model for Firmware aeb01a075 follow-up on DPU change api-ref da0d7494e Add ironic-grenade-skip-level Job d2039a29d Handle nova policy change 124ad571f Explicitly pin CIRROS_VERSION eaf26c7c6 Remove unused get_not_versions from dbapi ea68a4a79 Remove model_query use from general dbapi calls 93688e953 Explicitly use a session for DB version check 3f5e25e18 DPU modeling - parent_node DB/Model/API 27bf20911 [iRMC] Fix IPMI incompatibility handling error 4beeef777 CI: DB: Don't return inside of node get wrappers fce8c3a65 CI: Change tinycore URL 8b98dfafd CI: Disable mysql counters for grenade 1cb371327 CI: Try to isolate test failures in neutron vif logic 9c0b4c90a Fix Cinder Integration fallout from CVE-2023-2088 65b8895e8 Update docs: Ironic uses launchpad now 912dcbbdc CI: Mark BFV job non-voting for now c8c83ef54 Migrate to pysnmp lextudio ecosystem 995e38cd7 Don't return the in-flight SQL handler 9da6dfd73 Fix self_owned_node policy check 17d3b72b3 Remove indicators list by component from api-ref 3139460cd Imported Translations from Zanata 1b8c0be0b Fix api-ref v1-indicators cec72275a CI: Fix another network test 451857777 CI: Modify dhcp client ID fail b48dfd44c Use monotonic time for hashring reset cae05c70e Make rbac enforced test non-voting for the time being 7a5f80cec Fix anaconda stage2_id loading from image properties 03cd9788e Support longer checksums for redfish firmware upgrade 7f281392c Change wholedisk image checksum to sha256 f10958a54 Handle MissingAttributeError when using OOB inspections to fetch MACs c03a5b44e Remove autocommit, again. 75b881bd3 Fix DB/Lock session handling issues f2605e928 Remove use of nomodeset by default 3f09bdcf9 [iRMC] Fix parse_driver_info bug enforcing SNMP v3 under FIPS mode c7b8236ab Configure docs: we no longer use storyboard 510a612ee Add ablity to power off nodes in clean failed e0c0b771e [iRMC] Fix typo of Python string format in log message 153d4d838 Upgrade to latest hacking - v6 708354573 tests: Replace invalid UUIDs 00aadf570 db: Resolve SAWarning warnings 5268c50a7 Run metal3 integration with ironic-image from source a5a737e38 Set ironic-grenade to wait 120 seconds 9f422a8df Fix requests calls with timeouts cc9fa8526 Remove outdated API version information from the enrollment docs 59c6ad96c Always fall back from hard linking to copying files 4bcef0eee Do not log into the template1 database during test-setup ebc1053ce Exclude all files starting with . from flake8 tests 3e21560bf Remove all references to the "cpus" property b201e52f8 [CI] Fix port list benchmark d0448ea20 Use main branch of metal3-dev-env to run metal3 integration job 21437135a Add error logging on lookup failures in the API 6341003da Enables boot modes switching with Anaconda deploy for ilo driver c5e004a73 Fixes Secureboot with Anaconda deploy e30ba65f9 Refactoring: clean up inspection data handlers 3dd54a110 Refactoring: DRY in the root API controller 0e7c6f978 Refactoring: create ironic.conductor.inspection b13ce6eb3 Imported Translations from Zanata 48ea2c5e7 Update master for stable/2023.1 f00da959e Do not recalculate checksum if disk_format is not changed a6d87a608 Add a non-voting metal3 CI job bcf6c1226 Clean out agent token even if power is already off 47b590948 Wipe Agent Token when cleaning timeout occcurs 005f21c0d Fix auth_protocol and priv_protocol for SNMP v3 d43f7fbc6 Refactoring: extract some common functions from the inspector code 9acfd5136 Restructure the inspector module in preparation for its expansion b41d5d514 Document [fake] delay config values 6a9e319fb On rpc service stop, wait for node reservation release Diffstat (except docs and test files) ------------------------------------- README.rst | 2 +- api-ref/source/baremetal-api-v1-indicators.inc | 45 +--- api-ref/source/baremetal-api-v1-nodes.inc | 18 ++ api-ref/source/conf.py | 2 +- api-ref/source/parameters.yaml | 13 + .../node-indicators-component-list-response.json | 45 ---- .../source/samples/node-inventory-response.json | 1 - devstack/lib/ironic | 17 +- devstack/tools/ironic/scripts/cirros-partition.sh | 2 +- .../install/include/kernel-boot-parameters.inc | 4 +- driver-requirements.txt | 4 +- ironic/api/controllers/v1/__init__.py | 146 +++-------- ironic/api/controllers/v1/node.py | 206 +++++++++++---- ironic/api/controllers/v1/ramdisk.py | 6 +- ironic/api/controllers/v1/utils.py | 17 +- ironic/api/controllers/v1/versions.py | 4 +- ironic/common/cinder.py | 71 +++++- ironic/common/hash_ring.py | 4 +- ironic/common/image_service.py | 35 +-- ironic/common/keystone.py | 24 +- ironic/common/kickstart_utils.py | 4 +- ironic/common/molds.py | 6 +- ironic/common/policy.py | 12 +- ironic/common/pxe_utils.py | 3 +- ironic/common/release_mappings.py | 4 +- ironic/common/rpc_service.py | 18 +- ironic/common/utils.py | 28 ++ ironic/conductor/base_manager.py | 17 +- ironic/conductor/inspection.py | 108 ++++++++ ironic/conductor/manager.py | 100 +------- ironic/conductor/utils.py | 17 +- ironic/conf/conductor.py | 8 + ironic/conf/default.py | 5 +- ironic/conf/ilo.py | 2 +- ironic/conf/inventory.py | 15 +- ironic/conf/opts.py | 1 - ironic/conf/pxe.py | 2 +- ironic/conf/redfish.py | 2 +- ironic/db/api.py | 10 - ironic/db/sqlalchemy/__init__.py | 23 +- .../163040c5513f_add_firmware_information.py | 50 ++++ .../versions/fe222f476baf_add_parent_node_field.py | 35 +++ ironic/db/sqlalchemy/api.py | 269 ++++++++++++-------- ironic/db/sqlalchemy/models.py | 21 +- ironic/drivers/base.py | 4 +- .../ansible/playbooks/library/stream_url.py | 3 +- ironic/drivers/modules/deploy_utils.py | 37 ++- ironic/drivers/modules/drac/inspect.py | 14 - ironic/drivers/modules/drac/raid.py | 2 +- ironic/drivers/modules/ilo/boot.py | 12 + ironic/drivers/modules/inspect_utils.py | 129 +++++----- ironic/drivers/modules/inspector/__init__.py | 15 ++ ironic/drivers/modules/inspector/client.py | 57 +++++ .../{inspector.py => inspector/interface.py} | 105 ++------ ironic/drivers/modules/irmc/common.py | 12 + ironic/drivers/modules/irmc/inspect.py | 5 +- ironic/drivers/modules/irmc/management.py | 18 +- ironic/drivers/modules/pxe.py | 15 +- ironic/drivers/modules/pxe_base.py | 7 +- ironic/drivers/modules/redfish/firmware_utils.py | 18 +- ironic/drivers/modules/redfish/inspect.py | 4 +- ironic/drivers/modules/redfish/management.py | 11 +- ironic/drivers/modules/snmp.py | 4 +- ironic/hacking/checks.py | 2 +- ironic/objects/node.py | 8 +- .../unit/drivers/modules/drac/test_inspect.py | 22 -- .../unit/drivers/modules/inspector/__init__.py | 0 .../unit/drivers/modules/inspector/test_client.py | 65 +++++ .../test_interface.py} | 70 +---- .../unit/drivers/modules/irmc/test_inspect.py | 6 - .../unit/drivers/modules/network/test_common.py | 27 +- .../drivers/modules/redfish/test_firmware_utils.py | 24 ++ .../unit/drivers/modules/redfish/test_inspect.py | 18 +- .../drivers/modules/redfish/test_management.py | 10 + .../unit/drivers/modules/test_deploy_utils.py | 73 +++++- .../unit/drivers/modules/test_inspect_utils.py | 136 ++++------ playbooks/metal3-ci/fetch_kube_logs.yaml | 32 +++ playbooks/metal3-ci/fetch_pod_logs.yaml | 24 ++ playbooks/metal3-ci/post.yaml | 194 ++++++++++++++ playbooks/metal3-ci/run.yaml | 39 +++ .../Cleanfail-power-off-13b5fdcc2727866a.yaml | 8 + .../add-parent-node-support-10bd42abd008db6f.yaml | 23 ++ .../notes/bug-2010613-3ab1f32aaa776f28.yaml | 7 + .../notes/cinder-2019892-6b5a9de5c5f05aa6.yaml | 16 ++ .../notes/cross-link-1ffd1a4958f14fd7.yaml | 5 + ...b-sqlite-OperationalError-7934dbda2a21c69e.yaml | 6 + ...nforcing-snmpv3-with-fips-e45971d363925ec3.yaml | 6 + ...atibility-patch-situation-c246d2b59b2e8a78.yaml | 8 + .../fix-power-off-token-wipe-e7d605997f00d39d.yaml | 6 + ...ix-self-owned-node-policy-fc2dae357879dc33.yaml | 7 + ...a_deploy_with_ilo_drivers-16637adb62f0ed2f.yaml | 5 + ...boot_with_anaconda_deploy-84d7c1e3bbfa40f2.yaml | 4 + .../graceful_shutdown_wait-9a62627714b86726.yaml | 15 ++ ...g-ethernetinterfaces-attr-7e52f7259fe66762.yaml | 9 + releasenotes/notes/no-cpus-c79717303470bf3c.yaml | 6 + .../notes/no-recalculate-653e524fd6160e72.yaml | 5 + ...sh-firmware-sha256-sha512-3e40c3a087fe42b4.yaml | 6 + ...-model-query-from-upgrade-af227b6c8a5d654a.yaml | 9 + .../notes/remove-nomodset-7a352a9519c1045b.yaml | 22 ++ ...ken-upon-cleaning-timeout-c9add514fad1b02c.yaml | 7 + releasenotes/source/2023.1.rst | 6 + releasenotes/source/conf.py | 2 +- releasenotes/source/index.rst | 1 + .../locale/en_GB/LC_MESSAGES/releasenotes.po | 55 +++- test-requirements.txt | 4 +- tools/benchmark/generate-statistics.py | 2 + tools/test-setup.sh | 2 +- tox.ini | 4 +- zuul.d/ironic-jobs.yaml | 13 + zuul.d/metal3-jobs.yaml | 30 +++ zuul.d/project.yaml | 13 +- 159 files changed, 3129 insertions(+), 1473 deletions(-) Requirements updates -------------------- diff --git a/driver-requirements.txt b/driver-requirements.txt index 876e817cb..c5a90f2d0 100644 --- a/driver-requirements.txt +++ b/driver-requirements.txt @@ -8 +8,3 @@ proliantutils>=2.14.0 -pysnmp>=4.3.0,<5.0.0 +pysnmp-lextudio>=5.0.0 # BSD +pyasn1-lextudio>=1.1.0 # BSD +pyasn1-modules-lextudio>=0.2.0 # BSD diff --git a/test-requirements.txt b/test-requirements.txt index 0c4bdb0ca..57c4a9c2c 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -16 +16,3 @@ WebTest>=2.0.27 # MIT -pysnmp>=4.4.12 +pysnmp-lextudio>=5.0.0 # BSD +pyasn1-lextudio>=1.1.0 # BSD +pyasn1-modules-lextudio>=0.2.0 # BSD