Open Stack

We are overjoyed to announce the release of:

ironic 18.1.0: OpenStack Bare Metal Provisioning

This release is part of the xena release series.

The source is available from:

    https://opendev.org/openstack/ironic

Download the package from:

    https://tarballs.openstack.org/ironic/

Please report issues through:

    https://storyboard.openstack.org/#!/project/943

For more details, please see below.

18.1.0
^^^^^^


New Features
************

* The anaconda deploy interface now handles config drive. The config
  drive contents are written to the disk in the
  /var/lib/cloud/seed/config_drive directory by the driver via
  kickstart files %post section. cloud-init picks up the the config
  drive information and processes them. Because the config drive is
  extracted on to disk as plain text files, tools like glean do not
  work with this deploy interface.

* The configuration option "[deploy]ramdisk_image_download_source"
  now supports a new value "swift". If set, and if "boot_iso" or
  "deploy_iso" is a Glance image, the image is exposed via a Swift
  temporary URL. For other types of images the new value works the
  same way as the existing "http" value.

* The "ilo-virtual-media" deploy interface now supports "file:///"
  URLs for "boot_iso" and "deploy_iso".

* The "ilo-virtual-media" deploy interface now supports the
  "[deploy]ramdisk_image_download_source" configuration option.

* Adds "boot_mode" and "secure_boot" fields to node. These indicate
  the boot mode (bios/uefi) and secure boot state (True/False)
  detected in the most recent power sync or during transition to the
  "manageable" state. If underlying driver does not support detecting
  these, they shall be populated with null values. These fields are
  also available under a node's states endpoint: *
  "/v1/nodes/{node_ident}/states"

* For the "ramdisk" deploy interface, the
  "ramdisk_image_download_source" option can now be provided in the
  node's "instance_info" in addition to the global configuration.

* Provides new vendor passthru methods for Redfish to create, delete
  and get subscriptions for BMC events.


Upgrade Notes
*************

* The query pattern for the database when lists of nodes are
  retrieved has been changed to a more efficient pattern at scale,
  where a list of nodes is generated, and then additional queries are
  executed to composite this data together. This is from a model where
  the database client in the conductor was having to deduplicate the
  resulting data set which is overall less efficent.

* The default UEFI iPXE bootloader, "[pxe]uefi_ipxe_bootfile_name"
  used by the "ipxe" boot interface, has been changed from "ipxe.efi"
  to "snponly.efi". This is because most deployments actually need to
  use "snponly.efi" as it contains support for the UEFI integrated
  network stack, where as "ipxe.efi" does not and only contained
  compiled in network drivers, for UEFI, there happen to be few as the
  UEFI standard requires networking to be handled by the UEFI
  firmware.

* Since "ilo-virtual-media" deploy interface now respects the
  "[deploy]ramdisk_image_download_source" configuration options, its
  default caching behavior has changed. Now HTTP
  "boot_iso"/"deploy_iso" are cached locally and served from the
  conductor's HTTP server instead of passing them directly to the BMC.
  Glance images are also cached locally. To revert to the previous
  behavior, set the "[deploy]ramdisk_image_download_source" option to
  "swift".

* The minimum requirement for the "oslo.db" library is now version
  9.1.0 to address duplicate key error changes in MySQL 8.0.19.
  oslo.db version 9.1.0 fixes issue of idrac driver node inspection
  for inspect interface as idrac-redfish. Issue was introduced since
  MySQL 8.0.19 where duplicate key error information is extended to
  include the table name in the key.

  For more information, see story 2008901
  (https://storyboard.openstack.org/#!/story/2008901).


Deprecation Notes
*****************

* The "driver_info" property "ilo_boot_iso" has been renamed to just
  "boot_iso".

* The following "driver_info" parameters have been renamed with
  deprecation:

  * "ilo_deploy_kernel" -> "deploy_kernel"

  * "ilo_deploy_ramdisk" -> "deploy_ramdisk"

  * "ilo_deploy_iso" -> "deploy_iso"

  * "ilo_rescue_kernel" -> "rescue_kernel"

  * "ilo_rescue_ramdisk" -> "rescue_ramdisk"

  * "ilo_rescue_iso" -> "rescue_iso"

  * "ilo_bootloader" -> "bootloader"

* The "driver_info" properties "irmc_deploy_iso" and
  "irmc_rescue_iso" have been renamed to just "deploy_iso" and
  "rescue_iso".

* The "instance_info" property "irmc_boot_iso" has been renamed to
  just "boot_iso".

* The "[pxe]ip_version" setting has been deprecated and is
  anticipated to be removed in the Y* release of OpenStack. This
  option effectively has had no operational impact since the Ussuri
  release of OpenStack where dual stack IPv4 and IPv6 support was
  added to Ironic.


Critical Issues
***************

* Fixes upgrade failure caused by the missing version of
  "BIOSSetting" database objects.


Security Issues
***************

* Fixes an issue with the "/v1/nodes/detail" endpoint where an
  authenticated user could explicitly ask for an "instance_uuid"
  lookup and the associated node would be returned to the user with
  sensitive fields redacted in the result payload if the user did not
  explicitly have "owner" or "lessee" permissions over the node. This
  is considered a low-impact low-risk issue as it requires the API
  consumer to already know the UUID value of the associated instance,
  and the returned information is mainly metadata in nature. More
  information can be found in Storyboard story 2008976
  (https://storyboard.openstack.org/#!/story/2008976).


Bug Fixes
*********

* Skips port creation during redfish inspect for devices reported
  without a MAC address.

* Fixes potential cache coherency issues by caching the AgentClient
  per task, rather than globally.

* Fixes an issue with the "/v1/nodes/detail" endpoint where requests
  for an explicit "instance_uuid" match would not follow the standard
  query handling path and thus not be filtered based on policy
  determined access level and node level "owner" or "lessee" fields
  appropriately. Additional information can be found in story 2008976
  (https://storyboard.openstack.org/#!/story/2008976).

* Slow database retrieval of nodes has been addressed at the lower
  layer by explicitly passing and handling only the requested fields.
  The result is excess discarded work is not performed, making the
  overall process more efficent. This is particullarly beneficial for
  OpenStack Nova's syncronization with Ironic.

* Fixes configuring Redfish RAID using "interface_type" when error
  "failed to find matching physical disks for all logical disks"
  occurs.

* The "ilo-virtual-media" deploy interface no longer requires the
  Image service backend to be Swift for Glance images in "boot_iso"
  and "deploy_iso".

* Improves record retrieval performance for baremetal nodes by
  enabling ironic to not make redundant calls as part of generating
  API result sets for the baremetal nodes endpoint.

* Fixes handling of single-value (non-key-value) parameters in the
  "[inspector]extra_kernel_params" configuration options.

* Allows reboot to hard disk following iso ramdisk deploy. Re-uses
  the "force_persistent_boot_device" driver_info option which can be
  set to "Never" in order to set the virtual media to one time boot.

* The "ramdisk" deploy interface no longer requires a fake
  "image_source" value to be provided when "boot_iso" is not used.

* Removes unused local images after ejecting a virtual media device
  via the "eject_vmedia" vendor passthru call of the "redfish" vendor
  interface.

* In Redfish RAID clean and deploy steps skip non-RAID storage
  controllers for RAID operations. In Redfish systems that do not
  implement "SupportedRAIDTypes" they are still processed and could
  result in unexpected errors.

* Fixes an issue of powering off with the "idrac-wsman" management
  interface while the execution of a clear job queue cleaning step is
  proceeding. Prior to this fix, the clean step would fail when
  powering off a node.

* Fixes issue arose during inspection of iDRAC node with inspect-
  interface as idrac-redfish. Earlier, inspection of node was getting
  failed with error "port already exists". This issue arose since
  MySQL 8.0.19 where duplicate key error information is extended to
  include the table name in the key. Previously, duplicate key error
  information included only the key value and key name.

  For more information, see story 2008901
  (https://storyboard.openstack.org/#!/story/2008901).

* Fixes overriding "agent_verify_ca" with "False" via "driver_info".


Other Notes
***********

* The default database query pattern has been changed which will
  result in additional database queries when compositing lists of
  "nodes" by separately querying "traits" and "tags". Previously this
  was a joined query which requires deduplication of the result set
  before building composite objects.

* Deprecation warnings for the legacy RBAC policies are now
  suppressed, as the OpenStack community is coalescing around what
  appears will be a longer deprecation cycle and process than would
  typcially be undertaken due to the nature and impact of policy
  changes. The community as a whole is expecting to make RBAC policy
  work changes a community goal during the Y* release development
  cycle, which means the earliest legacy policy support may be removed
  is likely now the Z* development cycle.

Changes in ironic 18.0.0..18.1.0
--------------------------------

a2417571b Update vendor_passthru subscriptions releasenote
f095a9478 Prepare 18.1 and clean up release notes
4bc5142df Add vendor_passthru method for subscriptions
b5872c903 Set glance limit for baremetal friendly images
2cd646834 Scoped RBAC Devstack Plugin support
e488792cc Burn-in: Add documentation
58ca01c22 Add a section to redfish doc for BIOS registry fields
3199d289b Fix iPXE docs: snponly is not always available
06124bae7 Update requirements
e503048f0 Sanity check object version changes
3bad548ce Allow reboot to hard disk following iso ramdisk deploy.
ae535bac2 Add ipxe ramdisk kernel append param test
fdef42fee Update iDRAC virtual media boot known issue
46ff51487 Add `boot_mode` and `secure_boot` to node object and expose in api
7e5e65550 Add missing BIOSSettings version mapping for older releases
9e477d178 Fix typos in API sanitization change notes
d3737b23b Bring boot_iso/deploy_iso handling in iLO closer to Redfish
6bbd03b17 Use selectinload for all list queries
da27e370b Clean up images when ejecting an ISO with Redfish
9851b68ee Allow node_sanitize function to be provided overrides
c830861ee Avoid double file removal in create_boot_iso
85212fead Add note regarding configuration drives to tuning docs
8cb605745 Support "swift" for ramdisk_image_download_source
ad3ce797c Defer checking image size until instance info is built
34337f75f Fix oslo policy DeprecatedRule warnings
4ac8340fb Add support for configdrive in anaconda interface
24901e456 Add reno and reset legacy policy deprecation expectation
bb0523f03 [doc] Bootloader reinstallations on Software RAID
4e5073390 [doc] Update section on ESP consistency
c41ed8414 Fix Redfish RAID interface_type physical disk hint
67ff5ff2b Suppress policy deprecation and default change warnings
0cc6d8204 Skip port create if MAC is blank
d8ccfabda Ramdisk: do not require image_source
1d6441cc3 Refactor deploy_utils.validate_image_properties
c779be9f7 Redfish: Skip non-RAID controllers for RAID
b2a249d18 Deprecate [pxe]ip_version parameter
e3a4c7d77 Upgrade oslo.db version
741a4d8a8 Update the clear job id's constant
87e42afb9 API to pass fields to node object list
7b37e03b6 Set stage for objects to handle selected field lists.
644ba5d4b Only return the requested fields from the DB
fcb6a1096 Cache AgentClient on Task, not globally
4d13da9cc Change UEFI ipxe bootloader default
4dc192015 Refactor: untie IloVendor from validate_image_properties
ce5baa380 Fix handling driver_info[agent_verify_ca] == False
d7a5b3469 Fix ramdisk boot option handling
f8e2bc99d Allow ramdisk_image_download_source in instance_info for ramdisk deploy
ebb314bf4 Nicer error message when a deploy step fails
5a8a77ec5 Don't run the inspector job on changes to inspector tests
4f2d1ca94 CI: change ilo_deploy_iso to deploy_iso
be3c153d5 Fix node detail instance_uuid request handling
fc27710fa Clean up vendor prefixes for iRMC boot
595345ffc Add documentation for anaconda deploy interface
df228c5b9 Fix typos in inspection docs
f91915d4c Remove redundant/legacy is_admin logic
2c3470353 Split a community page out of the contributor docs
50b84c9ae Document managed inspection
fd7cde9b9 dhcp-less: mention how to provide network_data to instance
7507dec09 Use env to find python3
e84b6d102 Rename ilo_boot_iso -> boot_iso
349bf2d5b Clean up vendor prefixes for iLO boot
3824912f0 Handle non-key-value params in [inspector]extra_kernel_params
f58cbf151 Fix ironic-status db index check
728edb6b7 Changes made to enrollment documentation
0a8ebdfc5 Update Redfish RAID disk_type unit test
4b29f2b95 Redfish: Get only RAID controller's physical disks
96aae720a Improve agent_client logging
2fabd3b7d Trivial: fix an outdated link
6a4e33eb0 Move provision states documentation to the user guide
39cd751a9 Refactor iDRAC OEM extension manager calls
57120615a Update CI jobs list and description in docs


Diffstat (except docs and test files)
-------------------------------------

.../source/baremetal-api-v1-node-management.inc    |   7 +-
api-ref/source/parameters.yaml                     |  31 +-
.../source/samples/node-get-state-response.json    |   2 +
devstack/lib/ironic                                | 141 ++--
devstack/upgrade/upgrade.sh                        |  10 +-
.../contributor/ironic-multitenant-networking.rst  |   3 +
driver-requirements.txt                            |   2 +-
ironic/api/controllers/v1/collection.py            |  13 +-
ironic/api/controllers/v1/node.py                  | 379 +++++++----
ironic/api/controllers/v1/utils.py                 |   2 +
ironic/api/controllers/v1/versions.py              |   4 +-
ironic/api/hooks.py                                |   3 -
ironic/cmd/status.py                               |  31 +-
ironic/common/context.py                           |   4 -
ironic/common/glance_service/service_utils.py      |   7 +-
ironic/common/images.py                            |  11 +-
ironic/common/kickstart_utils.py                   | 165 +++++
ironic/common/policy.py                            | 743 ++++++++++-----------
ironic/common/pxe_utils.py                         |   9 +-
ironic/common/raid.py                              |  10 +
ironic/common/release_mappings.py                  |  36 +-
ironic/common/utils.py                             |  16 +
ironic/conductor/deployments.py                    |   6 +-
ironic/conductor/manager.py                        |   4 +
ironic/conductor/steps.py                          |   8 +-
ironic/conductor/utils.py                          |  44 ++
ironic/conf/deploy.py                              |   5 +-
ironic/conf/pxe.py                                 |   8 +-
ironic/db/api.py                                   |   6 +-
.../c1846a214450_add_boot_mode_and_secure_boot.py  |  32 +
ironic/db/sqlalchemy/api.py                        | 162 ++++-
ironic/db/sqlalchemy/models.py                     |   3 +
ironic/drivers/modules/agent.py                    |  17 +-
ironic/drivers/modules/agent_base.py               |  31 +-
ironic/drivers/modules/agent_client.py             |  40 +-
ironic/drivers/modules/ansible/deploy.py           |   6 +-
ironic/drivers/modules/boot_mode_utils.py          |   3 +
ironic/drivers/modules/deploy_utils.py             |  97 ++-
ironic/drivers/modules/drac/boot.py                |  67 +-
ironic/drivers/modules/drac/inspect.py             |  46 +-
ironic/drivers/modules/drac/management.py          | 187 +-----
ironic/drivers/modules/drac/raid.py                |  43 +-
ironic/drivers/modules/drac/utils.py               | 121 ++++
ironic/drivers/modules/ilo/boot.py                 | 261 +++-----
ironic/drivers/modules/ilo/power.py                |  15 +-
ironic/drivers/modules/ilo/vendor.py               |   7 +-
ironic/drivers/modules/image_utils.py              |  43 +-
ironic/drivers/modules/inspector.py                |   6 +-
ironic/drivers/modules/irmc/boot.py                |  72 +-
ironic/drivers/modules/pxe.py                      |  13 -
ironic/drivers/modules/pxe_base.py                 |  13 +-
ironic/drivers/modules/redfish/boot.py             |  34 +-
ironic/drivers/modules/redfish/raid.py             |  38 +-
ironic/drivers/modules/redfish/utils.py            |  38 +-
ironic/drivers/modules/redfish/vendor.py           | 192 ++++++
ironic/drivers/utils.py                            |  22 +-
ironic/objects/base.py                             |   6 +-
ironic/objects/node.py                             |  67 +-
.../unit/drivers/modules/drac/test_inspect.py      |  27 -
.../unit/drivers/modules/drac/test_management.py   | 128 +---
.../unit/drivers/modules/redfish/test_boot.py      |  71 +-
.../unit/drivers/modules/redfish/test_inspect.py   |  15 +
.../unit/drivers/modules/redfish/test_raid.py      | 213 +++++-
.../unit/drivers/modules/redfish/test_utils.py     | 203 ++++--
.../unit/drivers/modules/redfish/test_vendor.py    | 257 +++++--
.../unit/drivers/modules/test_agent_client.py      |  22 +
.../unit/drivers/modules/test_deploy_utils.py      | 134 ++--
.../unit/drivers/third_party_driver_mock_specs.py  |   3 +
.../notes/bios-version-dfd5c95805c295c5.yaml       |   5 +
releasenotes/notes/blank-mac-f8e0b37e0bd6d719.yaml |   5 +
...ache-agentclient-per-task-ec2231684e6876d9.yaml |   5 +
...ss-pattern-for-node-lists-a333dd9c5afa737d.yaml |  16 +
.../change-ipxe-default-file-a9ed5e17fc3d022e.yaml |  10 +
...upport-in-anaconda-deploy-f2aad59b4ff809ec.yaml |   9 +
...ed-instance-info-behavior-1375914a30621eca.yaml |  20 +
...-field-overhead-reduction-40be1821e38b468c.yaml |   9 +
.../notes/deploy-iso-swift-355ad6eba6c511b4.yaml   |   8 +
...dfish-raid-interface-type-4b3566b637cc2301.yaml |   5 +
.../notes/ilo-boot-iso-696b57f44f29852d.yaml       |   5 +
.../notes/ilo-deploy-iso-0c88edb5daff8a4e.yaml     |  22 +
.../notes/ilo-prefix-d5a65ac637f82f65.yaml         |  13 +
...ode-retrieval-performance-cf5a02eb629bf32c.yaml |   6 +
.../inspector-kernel-params-6db82c633d7361e2.yaml  |   5 +
.../notes/irmc-boot-iso-0e93f97acfa59778.yaml      |   8 +
.../notes/node-boot-mode-0662effa2a2644dc.yaml     |  10 +
.../notes/not-presistent-9c552f4209a84820.yaml     |   7 +
...isk-image-download-source-7a071aba7890ba95.yaml |   6 +
.../notes/ramdisk-validate-acbc4acdb71d10c2.yaml   |   5 +
.../notes/redfish-eject-iso-9875388ae09bc8f6.yaml  |   6 +
.../remove_pxe_ip_version-7a71f821ded28360.yaml    |   7 +
...-rbac-deprecation-for-now-779898e720a7bf4e.yaml |  10 +
.../skip-nonraid-controllers-f4a79e2c9e8080ce.yaml |   7 +
...clear-job-id-constant-fix-c69cf96c55364bb3.yaml |   7 +
...redfish_inspection_bugfix-b5edd35c337f80fc.yaml |  23 +
...dor-passthru-subscription-5d28a2420e2af111.yaml |   5 +
.../notes/verify-bool-ab3607429e005bed.yaml        |   4 +
requirements.txt                                   |  15 +-
test-requirements.txt                              |   6 +-
tools/check-releasenotes.py                        |   2 +-
zuul.d/ironic-jobs.yaml                            |  42 ++
zuul.d/project.yaml                                |   4 +
160 files changed, 5685 insertions(+), 2844 deletions(-)


Requirements updates
--------------------

diff --git a/driver-requirements.txt b/driver-requirements.txt
index e379bb99a..8b2af33a2 100644
--- a/driver-requirements.txt
+++ b/driver-requirements.txt
@@ -14 +14 @@ python-xclarityclient>=0.1.6
-sushy>=3.8.0
+sushy>=3.10.0
diff --git a/requirements.txt b/requirements.txt
index ca59703cc..9c4e1dc47 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4 +4 @@
-pbr!=2.1.0,>=2.0.0 # Apache-2.0
+pbr>=3.1.1 # Apache-2.0
@@ -19,2 +19,2 @@ oslo.config>=6.8.0 # Apache-2.0
-oslo.context>=2.19.2 # Apache-2.0
-oslo.db>=6.0.0 # Apache-2.0
+oslo.context>=2.22.0 # Apache-2.0
+oslo.db>=9.1.0 # Apache-2.0
@@ -24,2 +24,2 @@ oslo.middleware>=3.31.0 # Apache-2.0
-oslo.policy>=3.6.2 # Apache-2.0
-oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0
+oslo.policy>=3.7.0 # Apache-2.0
+oslo.serialization>=2.25.0 # Apache-2.0
@@ -32,2 +32,3 @@ pecan!=1.0.2,!=1.0.3,!=1.0.4,!=1.2,>=1.0.0 # BSD
-requests>=2.14.2 # Apache-2.0
-rfc3986>=0.3.1 # Apache-2.0
+pycdlib>=1.11.0 # LGPLv2
+requests>=2.18.0 # Apache-2.0
+rfc3986>=1.2.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 72cd8e6b9..bd29d9394 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -6 +6 @@ coverage!=4.4,>=4.0 # Apache-2.0
-ddt>=1.0.1 # MIT
+ddt>=1.2.1 # MIT
@@ -12 +12 @@ oslotest>=3.2.0 # Apache-2.0
-stestr>=1.0.0 # Apache-2.0
+stestr>=2.0.0 # Apache-2.0
@@ -17 +17 @@ pysnmp>=4.4.12
-bandit!=1.6.0,>=1.1.0,<2.0.0 # Apache-2.0
\ No newline at end of file
+bandit!=1.6.0,>=1.1.0,<2.0.0 # Apache-2.0