We are happy to announce the release of: ironic 24.0.0: OpenStack Bare Metal Provisioning This release is part of the caracal release series. The source is available from: https://opendev.org/openstack/ironic Download the package from: https://tarballs.openstack.org/ironic/ Please report issues through: https://storyboard.openstack.org/#!/project/943 For more details, please see below. 24.0.0 ^^^^^^ New Features ************ * Adds the capability to define a "default_conductor_group" setting which allows operators to assign a default conductor group to new nodes created in Ironic if they do not otherwise have a "conductor_group" set upon creation. By default, this setting has no value. * Adds support for Redfish based HTTPBoot, which leveragings the DMTF Redfish "HttpBootUri" "ComputerSystem" resource in a BMC, to assert the URL for the next boot operation. This requires Sushy 4.7.0 as the minimum version. * Adds a new capability allowing to attach or detach generic iso images as virtual media devices after a node has been provisioned. * Previously the key for building temporary URLs from Swift was taken from the *x-account-meta-temp-url-key* header in the object store account. Now the header *x-account-meta-temp-url-key-2* is also checked, which allows password rotation to occur without breaking old URLs. This applies to the following temporary URL scenarios: * Temp URL image transfer from Glance (when *[glance]swift_temp_url_key* is not set) * Publishing an image with the Swift publisher (*[redfish]use_swift=True* or *[ilo]use_web_server_for_images=False*) * Storing the config drive in Swift (*[deploy]configdrive_use_object_store=True*) * Fetching Swift stored firmware update payloads. * Introducing basic authentication and configurable authentication strategy support for image and image checksum download processes. This feature introduces 3 new configuration variables that could be used to select the authentication strategy and provide credentials for authentication strategies. The 3 variables are structured in way that 1 of them "[deploy]image_server_auth_strategy" (string) provides the ability to select between authentication strategies by specifying the name of the authentication strategy. Currently the only supported authentication strategy is the "http- basic" which will make IPA use HTTP(S) basic authentication also known as the "RFC 7617" standard. The other 2 variables are "[deploy]image_server_password" and "[deploy]image_server_user" provide username and password credentials for image download processes. The "[deploy]image_server_password" and "[deploy]image_server_user" are not strategy specific and could be reused for any username + password based authentication strategy, but for the moment these 2 variables are only used for the "http- basic" strategy. "[deploy]image_server_auth_strategy" doesn't just enable the feature but enforces checks on the values of the 2 related credentials. When the "http-basic" strategy is enabled for image server download workflow the download logic will make sure to raise an exception in case any of the credentials are None or an empty string. Example of activating the "http-basic" strategy can be found in *HTTP(s) Authentication strategy for user image servers* section of the admin guide. Upgrade Notes ************* * The Ironic service API Role Based Access Control policy has been updated to disable the legacy RBAC policy by default. The effect of this is that deprecated legacy roles of "baremetal_admin" and "baremetal_observer" are no longer functional by default, and policy checks may prevent actions such as viewing nodes when access rights do not exist by default. This change is a result of the new policy which was introduced as part of Secure Role Based Access Control effort along with the Consistent and Secure RBAC community goal and the underlying "[oslo_policy] enforce_scope" and "[oslo_policy] enforce_new_defaults" settings being changed to "True". The Ironic project believes most operators will observe no direct impact from this change, unless they are specifically running legacy access configurations utilizing the legacy roles for access. Operators which are suddenly unable to list or deploy nodes may have a misconfiguration in credentials, or need to allow the user's project the ability to view and act upon the node through the node "owner" or "lessee" fields. By default, the Ironic API policy permits authenticated requests with a "system" scoped token to access all resources, and applies a finer grained access model across the API for project scoped users. Ironic users who have not already changed their "nova-compute" service settings for connecting to Ironic may also have issues scheduling Bare Metal nodes. Use of a "system" scoped user is available, by setting "[ironic] system_scope" to a value of "all" in your nova-compute service configuration, which can be done independently of other services, as long as the credentials supplied are also valid with Keystone for system scoped authentication. Heat users which encounter any issues after this upgrade, should check their user's roles. Heat's execution and model is entirely project scoped, which means users will need to have access granted through the "owner" or "lessee" field to work with a node. Operators wishing to revert to the old policy configuration may do so by setting the following values in "ironic.conf".: [oslo_policy] enforce_new_defaults=False enforce_scope=False Operators who revert the configuration are encourated to make the necessary changes to their configuration, as the legacy RBAC policy will be removed at some point in the future in alignment with 2024.1-Release Timeline. Failure to do so will may force operators to craft custom policy override configuration. (https://specs.openstack.org/openstack/ironic-specs/specs/17.0 /secure-rbac.html) (https://docs.openstack.org/ironic/latest/configuration/sample- policy.html) (https://governance.openstack.org/tc/goals/selected /consistent-and-secure-rbac.html) (https://governance.openstack.org/tc/goals/selected/consistent-and- secure-rbac.html#id3) * Removes the sphinxcontrib-seqdiag dependency as the Pillow upgrade to version 10.x (from OpenStack upper constraints) breaks its usage. seqdiag has not been maintained for the last 3 years, hence the upgrade causes it to break. In the ironic docs (source) rst files, adds references to svg files, and keeps the svg files in the doc/source/images/ directory, alongside their associated .diag files as backup. * The default value of the configuration option "[inspector]require_managed_boot" is now "True" for the newer "agent" inspect interface. The older "inspector" implementation is not affected. Operators with deployments that support unmanaged inspection must set this value to "False" explicitly. * *python-swiftclient* is no longer a dependency, all OpenStack Swift operations are now down using *openstacksdk*. Configuration option *[swift]swift_max_retries* has been removed and any custom value will no longer have any effect on failed object- store operations. Deprecation Notes ***************** * The "deploy_kernel", "deploy_ramdisk", "rescue_kernel" and "rescue_ramdisk" configuration options, incorrectly deprecated in the 2023.2 release series, are no longer deprecated. * The "idrac" hardware type management interface steps "import_configuration" and "export_configuration" steps are deprecated, and will be removed once a formalized generic step templating mechanism has been created within Ironic. The Ironic community is open to reconsidering this decision should the overall bulk configuration reset/templating model become adopted by DMTF Redfish as a standardized cross-vendor feature. * The "ibmc" hardware type is deprecated due to a lack of upstream communication, driver maintenance, and a recognition that the Redfish hardware type likely works for the users at this point. This driver is expected to be removed during the "2024.2" development cycle. * The "xclarity" hardware type is deprecated due to a lack of upstream communication, driver maintenance, and a recognition that the Redfish hardware type is suitable for Lenovo hardware users moving forward. This driver is expected to be removed during the "2024.2" development cycle. * The "idrac-wsman" interfaces on the "idrac" hardware type are deprecated due to a lack of upstream communiation, and the decision of the driver's maintainer in the past to move in to the direction of using Redfish for driver interactions. These driver interfaces are expected to be removed during the "2024.2" development cycle. * Rootwrap support is deprecated since Ironic no longer runs any commands as root. Files "/etc/ironic/rootwrap.conf", "/etc/ironic/rootwrap.d" and the "ironic-rootwrap" command will be removed in a future release. Bug Fixes ********* * Firmware components are now also cached on the transition to the "manageable" state in addition to cleaning. This is consisent with how BIOS settings, vendor and boot mode are cached. * Fixes the behavior of "file:///" image URLs pointing at a symlink. Ironic no longer creates a hard link to the symlink, which could cause confusing FileNotFoundError to happen if the symlink is relative. * Nodes no longer get stuck in cleaning when the firmware components caching code raises an unexpected exception. * Prevents a database constraints error on caching firmware components when a supported component does not have the current version. * Fixes an issue when listing allocations as a project scoped user when the legacy RBAC policies have been disabled which forced an HTTP 406 error being erroneously raised. Users attempting to list allocations with a specific owner, different from their own, will now receive an HTTP 403 error. * In case the lldp raw data collected by the inspection process includes non utf-8 information, the parser fails breaking the inspection process. This patch works around that excluding the malformed data and adding an entry in the logs to provide information on the failed tlv. * Fixes an issue where a System Scoped user could not trigger a node into a "manageable" state with cleaning enabled, as the Neutron client would attempt to utilize their user's token to create the Neutron port for the cleaning operation, as designed. This is because with requests made in the "system" scope, there is no associated project and the request fails. Ironic now checks if the request has been made with a "system" scope, and if so it utilizes the internal credential configuration to communicate with Neutron. * When configured to listen on a unix socket, Ironic will now properly cleanup the unix socket on a clean service stop. * The "idrac" hardware type is now compatible with the "redfish" firmware interface. The link between them was missing initially. * Fixes the inspection lookup to consider all nodes with the same BMC hostname, as can happen with Redfish. In this case, the nodes are distinguished by MAC addresses. * Fixes getting details of a conductor if it uses a non-standard JSON RPC port or an IPv6 address as the name, e.g. "GET /v1/conductors/[2001:db8::1]:8090". Previously, it would result in a HTTP error 400. * Fixes "enable_netboot_fallback" to write out pxe config on adopt. * When configuring secure boot via Redfish, internal server errors are now retried for a longer period than by default, accounting for the SecureBoot resource unavailability during configuration on some hardware. * Fixes Raid creation issue in iLO6 and other BMC with latest schema by removing 'VolumeType', 'Encrypted' and changing placement of 'Drives' to inside 'Links'. * Fixes the payload format required to query physical storage drives using redfish, when configuring RAID using redfish. * Uses the volume_name provided in the target_raid_config field of a node to set the storage volume name when configuring RAID with the redfish driver (instead of discarding the volume_name given in target_raid_config) * Use the 'volume_name' field from the logical_disk in the target_raid_config field of a node, instead of just 'name' (which is incorrect as per the Ironic API expectation), to create the RAID volume using the Redfish driver Other Notes *********** * The classic "ilo" hardware types may be deprecated in the future for removal or major changes, however our last communication with the maintainers as of the "2024.1" Project Teams Gathering sessions indiated they were still working to determine their own forward path with a strong emphasis on the use of Redfish. Changes in ironic 23.1.0..24.0.0 -------------------------------- c6e055358 Remove dependency on pytz 4ac42cac1 More precise comment about when to disable v6 7705b0b2d Remove the sphinxcontrib-seqdiag dependency 32c9c7445 RedfishFirmwareInterface - Unit Tests & More logs 83001d62e typo: fix a typo in api configuration c7d490f84 Clean up removed services from devstack options 435932355 Disable legacy RBAC policy by default. 2e8db13e0 Flip require_managed_boot to True for the new agent inspection bf673c276 Account for nodes with the same BMC hostname in inspection lookup 7032a0d9a Stop using a specific mirror in infra 0b3ed093e Don't create a hardlink to a symlink when handling file:// URLs ccfefa477 Fix bug in devstack-guide: readd server create ebd8381db Corrected IP_ADDRESS to IP_VERSION 4398c11a5 Revert "Revert "RBAC: Fix allocation check"" to use Unauthorized 4b3163386 Revert "RBAC: Fix allocation check" fba57d595 Validate [deploy] image_server_auth_strategy 8dc1ecead [deploy] image_server_password should be secret d738df5aa Add df logs to metal3 integration job 84397f350 CI: Remove ironic-inspector-tempest-managed-non-standalone c901b15f6 RBAC: Fix allocation check c3074524d Fix system scoped manageable node network failure be09717be Drop rootwrap support 92eb54251 PXE configuration guide for unmanaged inspection 041a7d706 Redfish UefiHttp boot support 310603f3b Make bandit voting on check and gate a40e3fd5a docs: add distribution specific chain of trust warning around grub 4d3101940 Handle LLDP parse Unicode error b0b7ee425 Do not log lack of metrics support at WARNING lvl fe4844032 Update to latest pep8/code style versions c3296bba3 Fix typo in xclarity docs ce4d8a6bc [Docs] Adds changes to 'iLO' docs regarding Gen11 support 3489a0204 Fix versions in release notes 12cb5bc8b Remove unnecessary egg_info options 4cc167cc6 Test redfish with reduced sushy-tools feature set 607b8734e Cache firwmare components on the transition to "manageable" af8508f51 CI: use Swift for configdrive when available 9806258ed Revise ramdisk boot interface for clarity 2cdb09ba9 Revert "Move BFV job to non-voting" 6956b0619 Fixes Raid creation in iLO6 and other BMC with latest schema 6ffef5b16 Move BFV job to non-voting 23745d97f Fix two severe errors in the firmware caching code e67f71606 Fix GET for conductors with a port or IPv6 address 2373127c7 Documentation: tuning worker threads a98681f1d Fix volume_name issue in Redfish RAID 9c0996d1a First pass at the new in-band inspection docs bfaf64b46 Add volume name to Redfish RAID volumes 578c02813 Fix log message var reference a6e3a7f50 Handle internal server errors while configuring secure boot 31f0e17a1 fix nits related to image server bauth d6491b0c8 [trivial] remove note on non-voting snmp job 1ffdabed9 Change snmp job to not use a focal node 2db444bce Replace swiftclient usage with openstacksdk 25bb15aee Document wsgi_service fix from 16a806f 6e10ad9ad Add missing compatibility between idrac and redfish firmware 40edeed61 Improve logging in the dnsmasq DHCP provider facd862a3 Fix Redfish request collecting storage drives b44d16a15 Imported Translations from Zanata 5db871ead Deprecate configuration molds 203660a0b Fix *_by_arch documentation and un-deprecate the options without it 01507db18 Trivial: fix a typo in wsgi_service 090291221 Generic API for attaching/detaching virtual media 6ac130805 Multiple driver related deprecations ce300b3de CI: Remove deprecated devstack method 3ea4bb234 implement basic-auth support for user image download process eb65c0de9 Ensure enable_netboot_fallback writes out pxe config on adopt. 2e4811887 [trivial] add Python 3.11 mention in setup.cfg 9953b5a2e add default conductor group capability Diffstat (except docs and test files) ------------------------------------- .../baremetal-api-v1-attach-detach-vmedia.inc | 54 ++ api-ref/source/parameters.yaml | 20 +- .../source/samples/node-vmedia-attach-request.json | 4 + devstack/lib/ironic | 97 +-- devstack/upgrade/upgrade.sh | 2 +- .../images/agent-token-with-virtual-media.diag | 21 + .../images/agent-token-with-virtual-media.svg | 101 +++ .../glance-and-swift-for-partition-images.diag | 34 + .../glance-and-swift-for-partition-images.svg | 169 ++++ .../images/glance-and-swift-whole-disk-images.diag | 33 + .../images/glance-and-swift-whole-disk-images.svg | 162 ++++ etc/ironic/rootwrap.conf | 1 + etc/ironic/rootwrap.d/ironic-utils.filters | 6 +- ironic/api/controllers/v1/allocation.py | 13 +- ironic/api/controllers/v1/conductor.py | 8 +- ironic/api/controllers/v1/node.py | 111 ++- ironic/api/controllers/v1/utils.py | 5 + ironic/api/controllers/v1/versions.py | 4 +- ironic/common/args.py | 13 + ironic/common/boot_devices.py | 6 + ironic/common/exception.py | 6 + ironic/common/glance_service/image_service.py | 9 +- ironic/common/image_service.py | 80 +- ironic/common/images.py | 2 +- ironic/common/molds.py | 9 + ironic/common/neutron.py | 6 +- ironic/common/policy.py | 22 +- ironic/common/pxe_utils.py | 2 +- ironic/common/release_mappings.py | 4 +- ironic/common/swift.py | 151 ++-- ironic/common/utils.py | 39 +- ironic/common/wsgi_service.py | 2 +- ironic/conductor/cleaning.py | 5 +- ironic/conductor/manager.py | 122 ++- ironic/conductor/rpcapi.py | 55 +- ironic/conductor/utils.py | 56 +- ironic/conductor/verify.py | 8 +- ironic/conf/api.py | 2 +- ironic/conf/conductor.py | 20 +- ironic/conf/default.py | 9 +- ironic/conf/deploy.py | 24 + ironic/conf/inspector.py | 14 +- ironic/conf/swift.py | 13 +- ironic/dhcp/dnsmasq.py | 51 +- ironic/drivers/base.py | 26 + ironic/drivers/drac.py | 5 + ironic/drivers/modules/agent.py | 12 +- ironic/drivers/modules/deploy_utils.py | 8 - ironic/drivers/modules/drac/bios.py | 4 + ironic/drivers/modules/drac/management.py | 8 +- ironic/drivers/modules/drac/power.py | 4 + ironic/drivers/modules/drac/raid.py | 4 + ironic/drivers/modules/drac/vendor_passthru.py | 4 + ironic/drivers/modules/ibmc/management.py | 4 + ironic/drivers/modules/ibmc/power.py | 4 + ironic/drivers/modules/ibmc/raid.py | 4 + ironic/drivers/modules/ibmc/vendor.py | 4 + ironic/drivers/modules/inspect_utils.py | 60 +- ironic/drivers/modules/inspector/agent.py | 2 + .../drivers/modules/inspector/hooks/parse_lldp.py | 11 +- ironic/drivers/modules/inspector/interface.py | 11 +- ironic/drivers/modules/pxe_base.py | 3 +- ironic/drivers/modules/redfish/boot.py | 303 ++++++++ ironic/drivers/modules/redfish/firmware.py | 45 +- ironic/drivers/modules/redfish/firmware_utils.py | 15 +- ironic/drivers/modules/redfish/management.py | 34 +- ironic/drivers/modules/redfish/raid.py | 24 +- ironic/drivers/modules/redfish/utils.py | 11 + ironic/drivers/modules/redfish/vendor.py | 2 + ironic/drivers/modules/xclarity/management.py | 4 + ironic/drivers/modules/xclarity/power.py | 4 + ironic/drivers/redfish.py | 3 +- ironic/drivers/utils.py | 9 +- .../unit/api/controllers/v1/test_allocation.py | 11 + .../unit/api/controllers/v1/test_conductor.py | 13 + .../unit/drivers/modules/inspector/test_agent.py | 7 + .../unit/drivers/modules/redfish/test_boot.py | 850 +++++++++++++++++++++ .../unit/drivers/modules/redfish/test_firmware.py | 659 ++++++++++++++++ .../drivers/modules/redfish/test_firmware_utils.py | 16 +- .../drivers/modules/redfish/test_management.py | 60 +- .../unit/drivers/modules/redfish/test_raid.py | 68 +- .../unit/drivers/modules/test_inspect_utils.py | 49 +- playbooks/metal3-ci/post.yaml | 2 + ...d-default-conductor-group-a0355c5e9345a037.yaml | 7 + ...-redfish-httpboot-support-8d516158860c9d43.yaml | 7 + ...cache-firmware-components-485b3343ba1db5ee.yaml | 6 + ...hange-default-rbac-policy-f2f154043910f26a.yaml | 57 ++ .../notes/deploy-kernels-8998a9c301db483b.yaml | 6 + ...recate-idrac-config-molds-6ba6e557b11dd5ff.yaml | 10 + ...ulitple-driver-interfaces-e42e4fa1c960f596.yaml | 24 + .../notes/file-symlink-b65bd6b407bd1683.yaml | 6 + .../notes/firmware-fail-c6f6c70220373033.yaml | 8 + ...ocation-exception-on-list-c04e93fb9cace218.yaml | 8 + .../notes/fix-lldp-decode-00021e76db26b2a5.yaml | 9 + ...tem-scope-triggered-clean-22ada9b920c08365.yaml | 12 + .../fix-unix-socket-support-eaa0e350f4bfaf56.yaml | 3 + ...eric-virtual-media-attach-9625f8ac66093b76.yaml | 5 + .../notes/idrac-firmware-3839648d729d9c7c.yaml | 5 + .../notes/lookup-many-bmcs-b019f3599c8e8da7.yaml | 6 + releasenotes/notes/no-root-8127c35b4702d242.yaml | 6 + .../notes/port-in-conductor-a354a2665effca2e.yaml | 7 + .../notes/pxe-onadopt-7214eba4f5822e1a.yaml | 5 + .../notes/redfish-500-fea3a8f86c0aecc7.yaml | 6 + ...redfish-fix-raid-creation-f437066b1301c032.yaml | 6 + ...dfish-raid-get-drives-fix-18d46f3e7275b0ef.yaml | 5 + ...fish-raid-set-volume-name-76205f8bb7dd2bb8.yaml | 6 + ...fish-raid-volume-name-fix-187c1b3e9f89cff3.yaml | 7 + ...remove-seqdiag-dependency-bfe9daea763dc0a3.yaml | 9 + .../require-managed-boot-c33e8aa9cba1502c.yaml | 8 + .../notes/temp_url_key_rot-1e7cb004df8c788f.yaml | 25 + ...r-image-server-basic-auth-c2b605aade241901.yaml | 32 + releasenotes/source/2023.2.rst | 6 +- .../locale/en_GB/LC_MESSAGES/releasenotes.po | 5 +- .../source/locale/ja/LC_MESSAGES/releasenotes.po | 159 ---- requirements.txt | 5 +- setup.cfg | 8 +- tox.ini | 10 +- zuul.d/ironic-jobs.yaml | 46 +- zuul.d/metal3-jobs.yaml | 1 + zuul.d/project.yaml | 20 +- 170 files changed, 6228 insertions(+), 1308 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index aa0e212b3..f87070972 100644 --- a/requirements.txt +++ b/requirements.txt @@ -18,2 +17,0 @@ ironic-lib>=5.5.0 # Apache-2.0 -python-swiftclient>=3.2.0 # Apache-2.0 -pytz>=2013.6 # MIT @@ -24,0 +23 @@ oslo.db>=9.1.0 # Apache-2.0 +# TODO(dtantsur): remove rootwrap when we no longer provide ironic-rootwrap CLI @@ -50 +49 @@ openstacksdk>=0.48.0 # Apache-2.0 -sushy>=4.3.0 +sushy>=4.7.0