We are jazzed to announce the release of: ironic 26.1.0 This release is part of the dalmatian release series. The source is available from: https://opendev.org/openstack/ironic Download the package from: https://tarballs.openstack.org/ironic/ Please report issues through: https://bugs.launchpad.net/ironic/+bugs For more details, please see below. 26.1.0 ^^^^^^ Prelude ******* The Ironic project is pleased to announce the release Ironic 26.1, also known as the "2024.2" or "Dalmatian" release. This release adds functionality to enable the creation and use of "runbooks" of repeated steps in "cleaning" and "service" states. Furthermore, cloud admins are now able to "lease" baremetal nodes to Nova users automatically through the use of metadata provided by Nova. Firmware updates have improved capability and ease-of-use on BMCs which manage multiple hosts. Also users of the "redfish" hardware type can now invoke firmware updates as a "service" step. Operators seeking to migrate away from BIOS booting now have additional options to help enforce migration mandates. Coupled with numerous bug fixes and other minor enhancements, this release of Ironic will improve the life of operators for years to come. We hope you enjoy! New Features ************ * Adds a new API concept, runbooks, to enable self-service of maintenance items on nodes by project members. Runbooks are curated lists of steps that can be run on nodes only associated via traits and used in lieu of an explicit list of steps for manual cleaning or servicing. * Adds a new top-level REST API endpoint */v1/runbooks/* with basic CRUD support. * Extends the */v1/nodes/<node>/states/provision* API to accept a runbook ident (name or UUID) instead of *clean_steps* or *service_steps* for servicing or manual cleaning. * Implements RBAC-aware lifecycle management for runbooks, allowing projects to limit who can CRUD and use a runbook. * Ironic now supports automatically setting node.lessee at deployment time using metadata provided at deploy time, typically by OpenStack Nova. When "[conductor]/automatic_lessee_source" is set to "instance", Ironic will set the lessee field on the node and remove it before cleaning. * Adds a new capability allowing to fetch the list of virtual media devices attached to a node by making a GET request. * Make the "idrac" hardware type inherit from the "redfish" hardware type since the "idrac" hardware type is an extension of the "redfish" with Dell specific overrides. This will ensure that features available to the "redfish" hardware type will always be available to "idrac". Added "redfish" interface as available for the "bios", "power" and "vendor" interfaces of the "idrac" hardware type. Upgrade Notes ************* * When upgrading Ironic to address the "qemu-img" image conversion security issues, the "ironic-python-agent" ramdisks will also need to be upgraded. * When upgrading Ironic to address the "qemu-img" image conversion security issues, the "[conductor]conductor_always_validates_images" setting may be set to "True" as a short term remedy while "ironic- python-agent" ramdisks are being updated. Alternatively it may be advisable to also set the "[agent]image_download_source" setting to "local" to minimize redundant network data transfers. * As a result of security fixes to address "qemu-img" image conversion security issues, a new configuration parameter has been added to Ironic, "[conductor]permitted_image_formats" with a default value of "raw,qcow2,iso". Raw and qcow2 format disk images are the image formats the Ironic community has consistently stated as what is supported and expected for use with Ironic. These formats also match the formats which the Ironic community tests. Operators who leverage other disk image formats, may need to modify this setting further. * "[conductor]/automatic_lessee" has been deprecated in favor of "[conductor]/automatic_lessee_source". Standalone Ironic deployments previously setting "automatic_lessee" to "True" now may want to set "automatic_lessee_source" to "request" to retain existing behavior. Deployers explicitly setting "automatic_lessee" to false may want to set "automatic_lessee_source" to "none" to retain existing behavior. The old configuration option, when explicitly set, will be honored until fully removed. * Ironic will now automatically set the node.lessee field for all deployments by default when provided in node instance_info at deployment time. Deployers are encouraged to review their security settings and Ironic Secure RBAC documentation to ensure no unexpected access is granted. * Ironic now requires rescue passwords to be hashed. Operators who would like to continue using unhashed passwords must set *[conductor]/require_rescue_password_hashed* to "false". Deprecation Notes ***************** * The "[[agent]]manage_agent_boot" configuration directive is being deprecated. It is completely untested, and requires operators to manually configure significant parts of infrastructure typically handled by Ironic. Operators using this configuration are advised to migrate away from it before it's scheduled removal during the 2025.2 cycle releases, coming out late 2025. * Deprecates the "idrac-redfish" interfaces in favor of the "redfish" interfaces for the "bios", "power", and "vendor" interfaces. This is a no-op change as these interfaces wrapped the "redfish" interface with no change already. Security Issues *************** * Ironic now checks the supplied image format value against the detected format of the image file, and will prevent deployments should the values mismatch. If being used with Glance and a mismatch in metadata is identified, it will require images to be re-uploaded with a new image ID to represent corrected metadata. This is the result of CVE-2024-44082 tracked as bug 2071740 (https://bugs.launchpad.net/ironic/+bug/2071740). * Ironic *always* inspects the supplied user image content for safety prior to deployment of a node should the image pass through the conductor, even if the image is supplied in "raw" format. This is utilized to identify the format of the image and the overall safety of the image, such that source images with unknown or unsafe feature usage are explicitly rejected. This can be disabled by setting "[conductor]disable_deep_image_inspection" to "True". This is the result of CVE-2024-44082 tracked as bug 2071740 (https://bugs.launchpad.net/ironic/+bug/2071740). * Ironic can also inspect images which would normally be provided as a URL for direct download by the "ironic-python-agent" ramdisk. This is not enabled by default as it will increase the overall network traffic and disk space utilization of the conductor. This level of inspection can be enabled by setting "[conductor]conductor_always_validates_images" to "True". Once the "ironic-python-agent" ramdisk has been updated, it will perform similar image security checks independently, should an image conversion be required. This is the result of CVE-2024-44082 tracked as bug 2071740 (https://bugs.launchpad.net/ironic/+bug/2071740). * Ironic now explicitly enforces a list of permitted image types for deployment via the "[conductor]permitted_image_formats" setting, which defaults to "raw", "qcow2", and "iso". While the project has classically always declared permissible images as "qcow2" and "raw", it was previously possible to supply other image formats known to "qemu-img", and the utility would attempt to convert the images. The "iso" support is required for "boot from ISO" ramdisk support. * Ironic now explicitly passes the source input format to executions of "qemu-img" to limit the permitted qemu disk image drivers which may evaluate an image to prevent any mismatched format attacks against "qemu-img". * The "ansible" deploy interface example playbooks now supply an input format to execution of "qemu-img". If you are using customized playbooks, please add "-f {{ ironic.image.disk_format }}" to your invocations of "qemu-img". If you do not do so, "qemu-img" will automatically try and guess which can lead to known security issues with the incorrect source format driver. * Operators who have implemented any custom deployment drivers or additional functionality like machine snapshot, should review their downstream code to ensure they are properly invoking "qemu-img". If there are any questions or concerns, please reach out to the Ironic project developers. * Operators are reminded that they should utilize cleaning in their environments. Disabling any security features such as cleaning or image inspection are at **your** **own** **risk**. Should you have any issues with security related features, please don't hesitate to open a bug with the project. * The "[conductor]disable_deep_image_inspection" setting is conveyed to the "ironic-python-agent" ramdisks automatically, and will prevent those operating ramdisks from performing deep inspection of images before they are written. * The "[conductor]permitted_image_formats" setting is conveyed to the "ironic-python-agent" ramdisks automatically. Should a need arise to explicitly permit an additional format, that should take place in the Ironic service configuration. Bug Fixes ********* * Adds microversion headers to the root ('/') endpoint. * Fixes multiple issues in the handling of images as it relates to the execution of the "qemu-img" utility, which is used for image format conversion, where a malicious user could craft a disk image to potentially extract information from an "ironic-conductor" process's operating environment. Ironic now explicitly enforces a list of approved image formats as a "[conductor]permitted_image_formats" list, which mirrors the image formats the Ironic project has historically tested and expressed as known working. Testing is not based upon file extension, but upon content fingerprinting of the disk image files. This is tracked as CVE-2024-44082 via bug 2071740 (https://bugs.launchpad.net/ironic/+bug/2071740). * Fixes inspection failure when "bmc_address" or "bmc_v6address" is "null" in the inventory received from the ramdisk. * The network_data fetched from Neutron contained 'links', 'networks' but was missing 'services'. This patch brings in 'services' to include dns nameservers that can be configured by Glean or cloud- init during cleaning and provisioning operations, especially when virtual media boot is used without DHCP. * Set node "alive" and make it fast trackable as soon as inspection is finished, in addition add a wait for the agent to callback should it not be available when fast track is attempted. * Replaces *ari/aki* format references with appropriate artifacts for kernel and ramdisk in the documentation and Ironic DevStack plugin. * Update "kernel_append_params" to match the "[pxe]" configuration, addressing a TODO from the Xena release. Other Notes *********** * Removes support for pre-SQLAlchemy 2.0 query objects from the internal database API. Downstream plug-ins must be adjusted to use the new-style queries. Changes in ironic 26.0.0..26.1.0 -------------------------------- 5857ca0cd Update release mappings f082c24d2 Add Prelude for end of cycle release eb27c6e21 Drop SQLALCHEMY_WARN_20 a8c3beaca Bring back the metal3-integration job b19507d64 Add microversion headers to root ('/') endpoint 1c201f094 Make sure qemu-img command is available in debian/suse 771cee464 Drop description for ZeroMQ d44a65883 Remove legacy Query support from _paginate_query 4fea3575c Fix inspection if bmc_address or bmc_v6address is None 510f87a03 Try limiting MTU to at least 1280 3f6507d85 Remove default override for RBAC config options 94fcae5e1 CI: Remove scope enforced ci jobs c996aafa6 CVE-2024-44982: Harden all image handling and conversion code b07ebc412 Set node "alive" when inspection finished e01522cd4 Imported Translations from Zanata 40e100387 Remove skip check for Python 3.6 0eda3d65e Support Automatic Lessee from instance metadata 42bd2a069 Use the correct command to fetch ovs state 52fdbb743 idrac: inherit driver interface from redfish 72c792bb2 Temporarily disable PXE booting test in the standalone job aaec00275 Replace reference to ari/aki 1aa780377 Deduplicate and remove invalid information for steps df39b7874 Fix versions in release notes 11dfe23cf Update network_data.json to include dns nameservers 8e6b758bf Clarify how to configure for non-master branch e3d6b644f Enable the inspection test in the redfish standalone job 17ba2b3db docs-audit-2024: Labeling references 018a7dcae Better handle missing inspection_network 2a6abc810 [docs] Add procedure to EoL and delete old bugfix branch c316443c9 Some documentaion improvements for ironic docs 040924ee6 Add unit test to ensure full node object is not logged a885a0544 Update configuration value in iRMC 754cf3f58 Fix conductor startup warning message ba096e091 Follow up to the runbooks change (#922142) a9a6ec7b3 Update error message 582b2e991 Link to configuration options 23b61e2ba Fix error message not interpolated 3f34f04bf [CI] Fix job parent name c9cf2347e add virtual media GET api 4750c4e9f docs-audit-2024: Use gerunds for task headings db7e4deff DevStack: enable the new in-band inspection by default 48f50248c Self-Service via Runbooks 816a7dfe8 Remove ironic-inspector from bugfix branches bfdfd6c16 [devstack] Ensure downgrade_dnsmasq is idempotent 631f50d21 CI: set tftp folder permission d146558ac Require hashed passwords for rescue by default 465bedbe7 Deprecate [agent]/manage_agent_boot 223a70043 Test devstack OSCaas against all Ironic jobs Diffstat (except docs and test files) ------------------------------------- api-ref/source/baremetal-api-v1-get-vmedia.inc | 21 + .../source/baremetal-api-v1-node-management.inc | 18 + api-ref/source/baremetal-api-v1-runbooks.inc | 245 +++++ api-ref/source/index.rst | 1 + api-ref/source/parameters.yaml | 67 ++ .../source/samples/node-set-provision-state.json | 4 + api-ref/source/samples/node-set-service-state.json | 12 + api-ref/source/samples/runbook-create-request.json | 19 + .../source/samples/runbook-create-response.json | 34 + .../source/samples/runbook-detail-response.json | 39 + api-ref/source/samples/runbook-list-response.json | 18 + api-ref/source/samples/runbook-show-response.json | 35 + api-ref/source/samples/runbook-update-request.json | 7 + .../source/samples/runbook-update-response.json | 34 + bindep.txt | 4 +- devstack/lib/ironic | 58 +- devstack/settings | 11 +- ironic/api/controllers/root.py | 2 + ironic/api/controllers/v1/__init__.py | 9 +- ironic/api/controllers/v1/deploy_template.py | 38 +- ironic/api/controllers/v1/node.py | 90 +- ironic/api/controllers/v1/notification_utils.py | 3 + ironic/api/controllers/v1/ramdisk.py | 6 +- ironic/api/controllers/v1/runbook.py | 391 +++++++ ironic/api/controllers/v1/utils.py | 142 +++ ironic/api/controllers/v1/versions.py | 6 +- ironic/cmd/conductor.py | 2 +- ironic/common/exception.py | 20 + ironic/common/image_format_inspector.py | 1038 ++++++++++++++++++ ironic/common/images.py | 152 ++- ironic/common/lessee_sources.py | 28 + ironic/common/neutron.py | 17 +- ironic/common/policy.py | 145 ++- ironic/common/qemu_img.py | 89 ++ ironic/common/release_mappings.py | 75 +- ironic/conductor/cleaning.py | 6 + ironic/conductor/deployments.py | 100 +- ironic/conductor/manager.py | 27 +- ironic/conductor/rpcapi.py | 21 +- ironic/conductor/utils.py | 21 +- ironic/conf/__init__.py | 2 + ironic/conf/agent.py | 3 +- ironic/conf/conductor.py | 84 +- ironic/conf/default.py | 4 +- ironic/conf/disk_utils.py | 33 + ironic/conf/irmc.py | 3 +- ironic/conf/opts.py | 1 + ironic/db/api.py | 95 ++ .../66bd9c5604d5_add_runbook_and_runbook_step.py | 70 ++ ironic/db/sqlalchemy/api.py | 271 ++++- ironic/db/sqlalchemy/models.py | 45 + ironic/drivers/base.py | 10 + ironic/drivers/drac.py | 45 +- .../playbooks/roles/deploy/tasks/write.yaml | 2 +- ironic/drivers/modules/deploy_utils.py | 202 +++- ironic/drivers/modules/drac/bios.py | 2 + ironic/drivers/modules/drac/power.py | 3 +- ironic/drivers/modules/drac/vendor_passthru.py | 2 + ironic/drivers/modules/image_cache.py | 71 +- ironic/drivers/modules/pxe_base.py | 5 +- ironic/drivers/modules/redfish/boot.py | 82 ++ ironic/drivers/modules/redfish/inspect.py | 2 +- ironic/drivers/modules/redfish/management.py | 12 + ironic/drivers/modules/storage/external.py | 6 +- ironic/objects/__init__.py | 1 + ironic/objects/runbook.py | 252 +++++ .../common/json_samples/neutron_subnet_show.json | 2 +- .../json_samples/neutron_subnet_show_ipv6.json | 2 +- .../drivers/modules/redfish/test_management.py | 6 + .../unit/drivers/modules/test_deploy_utils.py | 515 ++++++++- playbooks/ci-workarounds/get_extra_logging.yaml | 5 +- .../notes/2024.2-prelude-66649e7ee0781484.yaml | 15 + ...-headers-to-root-endpoint-199cb910a7aa53f9.yaml | 4 + .../notes/add-runbooks-38c3efa97ace8c67.yaml | 19 + .../address-qemu-issues-1bbead8bb70b76fb.yaml | 108 ++ .../automatic-lessee-source-37abe917b8cb5c36.yaml | 24 + .../notes/bmc-address-none-8214a3f7b67d6a0d.yaml | 5 + ...precate-manage-agent-boot-8474287a11b485cc.yaml | 9 + ...work-data-dns-nameservers-f363b3a66c109b4d.yaml | 9 + ...generic-virtual-media-get-f09003e5031b9c3d.yaml | 5 + .../idrac-extends-redfish-6692c2ecdb07dc2e.yaml | 15 + ...re-hashed-rescue-password-6f7c0424e12c1aeb.yaml | 5 + ...-when-inspection-finished-1ec74828852eaeef.yaml | 7 + .../notes/sqla-query-d5347ae962b06428.yaml | 6 + ...isk-artifacts-delineation-15a7d652d34b196f.yaml | 5 + ...of-a-configuration-option-91708f3366afd4c6.yaml | 5 + releasenotes/source/2023.2.rst | 2 +- releasenotes/source/2024.1.rst | 6 +- releasenotes/source/icehouse.rst | 6 +- .../locale/en_GB/LC_MESSAGES/releasenotes.po | 65 +- requirements.txt | 2 +- tox.ini | 2 - zuul.d/ironic-jobs.yaml | 7 +- zuul.d/project.yaml | 7 - 156 files changed, 9052 insertions(+), 628 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 7773d6e40..25f9d4c49 100644 --- a/requirements.txt +++ b/requirements.txt @@ -22 +22 @@ oslo.middleware>=3.31.0 # Apache-2.0 -oslo.policy>=3.7.0 # Apache-2.0 +oslo.policy>=4.4.0 # Apache-2.0