Open Stack

We are tickled pink to announce the release of:

ironic 22.0.0: OpenStack Bare Metal Provisioning

This release is part of the bobcat release series.

The source is available from:

    https://opendev.org/openstack/ironic

Download the package from:

    https://tarballs.openstack.org/ironic/

Please report issues through:

    https://storyboard.openstack.org/#!/project/943

For more details, please see below.

22.0.0
^^^^^^


New Features
************

* Add new conductor conf option: [conductor]poweroff_in_cleanfail
  (default: False). when True nodes entering clean failed state will
  be powered off. This option may be unsafe when using Cleaning to
  perform hardware-transformative actions such as firmware upgrade.

* Adds the concept of "parent_node" which allows a "child node",
  such as an independently managed BMC controlled device deployed
  within a "parent_node" as part of API version *1.83*. Child nodes
  are hidden from normal node lists as they are not "general purpose"
  machines, but have a specific embedded usage. In this model, RBAC
  rules also apply so if you wish an "owner" or "lessee" to have the
  child node visible, they must also have the the appropriate "owner"
  or "lessee" value set matching the parent node.

* Adds a "/v1/nodes/?include_children=True" parameter to get a list
  of all nodes and their children.

* Adds a "/v1/nodes/?parent_node=<node_ident>" query parameter to
  permit retrieval of a list of child nodes assigned to the parent
  denoted by "<node_ident>".

* On shutdown the conductor will wait for at most
  "[DEFAULT]graceful_shutdown_timeout" seconds for existing lock node
  reservations to clear. Previously lock reservations were cleared
  immediately, which in some cases would result in nodes going into a
  failed state.

* The Redfish firmware upgrade interface now supports checksum
  determination by length, and "sha256" and "sha512" checksums may now
  be supplied to the step arguments.


Upgrade Notes
*************

* This upgrade contains an additional field for the "nodes" table,
  named "parent_node". This update also indexes the "parent_node"
  database column to prevent performance issues in large deployments.

* "[DEFAULT]graceful_shutdown_timeout" defaults to 60s. Systemd
  "TimeoutStopSec" defaults to 30s. Kubernetes
  "terminationGracePeriodSeconds" defaults to 90s. It is recommended
  to align the value of "[DEFAULT]graceful_shutdown_timeout" with the
  graceful timeout of the process manager of the conductor process.

* Fully removes the "cpus" property from the documentation and
  inspect interface implementations. It was never used internally by
  Ironic, and is no longer used by Nova.

* The defaults for "kernel_append_params" have had the Linux kernel
  command line parameter "nomodeset" removed from the defaults for the
  "kernel_append_params" settings. The "nomodeset" option is for
  troubleshooting and changes the behavior of the graphics interface
  such that memory can be locked upon graphical updates on physical
  servers with BMC graphical interfaces, which results in spikes in
  latency and packet loss whenever graphics updates occur. Operators
  may add the option to their local configuration, but should be aware
  that large image transfers or other high IO operations can be
  impacted.


Bug Fixes
*********

* [bug 2010613 (https://storyboard.openstack.org/#!/story/2010613)]
  Fixes issue with SNMP v3 auth protocol and priv protocol set in
  driver info not being retrieved correctly when a SNMP client is
  initialized.

* Fixes Ironic integration with Cinder because of changes which
  resulted as part of the recent Security related fix in bug 2004555
  (https://launchpad.net/bugs/2004555). The work in Ironic to track
  this fix was logged in bug 2019892
  (https://bugs.launchpad.net/ironic/+bug/2019892). Ironic now sends a
  service token to Cinder, which allows for access restrictions added
  as part of the original CVE-2023-2088 fix to be appropriately
  bypassed. Ironic was not vulnerable, but the restrictions added as a
  result did impact Ironic's usage. This is because Ironic volume
  attachments are not on a shared "compute node", but instead mapped
  to the physical machines and Ironic handles the attachment life-
  cycle after initial attachment.

* Fixes "Invalid cross-device link" in some cases when using
  "file://" image URLs.

* Fixes issues in Ironic's use of SQLAlchemy with SQLite Databases,
  which is common with users like Metal3, which prevented Ironic from
  supporting SQLAlchemy 2.0 properly, as autocommit was re-enabled.

* Fixes bug of iRMC driver in parse_driver_info where, if FIPS is
  enabled, SNMP version is always required to be version 3 even though
  iRMC driver's xxx_interface doesn't use SNMP actually.

* Fixes bug in iRMC driver, where *irmc* power_interface sets and
  updates *irmc_ipmi_succeed* flag which is used by rest of iRMC
  driver code to deal with iRMC firmware's IPMI incompatibility but
  *ipmitool* power_interface doesn't set nor update
  *irmc_ipmi_succeed* flag and rest of iRMC driver code fail to handle
  iRMC firmware's IPMI incompatibility correctly.

* Fixes an issue where an agent token could be inadvertently
  orphaned if a node is already in the target power state when we
  attempt to turn the node off.

* Fixes scope classification check with the "self_owned_node" policy
  check where it was limited to check execution with only project
  scoped, so system scoped users who ticked the policy endpoint would
  basically get an incorrect error.

* Enables boot mode switching during anaconda deploy for "ilo" and
  "ilo5" hardware types.

* Fixes secure boot with anaconda deploy.

* Fixes the bug where provisioning a Redfish managed node fails if
  the BMC doesn't support EthernetInterfaces attribute, even if MAC
  address information is provided manually. This is done by handling
  of MissingAttributeError sushy exception in get_mac_addresses()
  method. This fix is needed to successfully provision machines such
  as Cisco UCSB and UCSX.

* No longer re-calculates checksums for images that are already raw.
  Previously, it would cause significant delays in deploying raw
  images.

* Fixes an issue where the database upgrade can hang on Python 3.10.
  This was because open transactions could become orphaned awaiting
  the Python runtime to clean up their memory references due to the
  way the overall database query was being intiiated to pre-flight
  check the upgrade. We have structurally changed the behavior to
  remedy this case.

* Agents deploying on physical servers with default kernel arguments
  were suspetible to packet loss if a Matrox VGA/Aspeed BMC Graphics
  interface is present on the machine. The defaults have been changed
  to remove the use of the "nomodeset" kernel command line parameter
  which should only be used for troubleshooting as it has been
  determined that the memory updates can lock all of the kernel memory
  upon any console graphics update which can negatively impact IO for
  Networking or Disk interactions.

* Fixes an issue where an agent token was being orphaned if a
  baremetal node timed out during cleaning operations, leading to
  issues where the node would not be able to establish a new token
  with Ironic upon future in some cases. We now always wipe the token
  in this case.

Changes in ironic 21.4.0..22.0.0
--------------------------------

bf850cad1 Make metal3 job voting
d66530494  Add DB model for Firmware
aeb01a075 follow-up on DPU change api-ref
da0d7494e Add ironic-grenade-skip-level Job
d2039a29d Handle nova policy change
124ad571f Explicitly pin CIRROS_VERSION
eaf26c7c6 Remove unused get_not_versions from dbapi
ea68a4a79 Remove model_query use from general dbapi calls
93688e953 Explicitly use a session for DB version check
3f5e25e18 DPU modeling - parent_node DB/Model/API
27bf20911 [iRMC] Fix IPMI incompatibility handling error
4beeef777 CI: DB: Don't return inside of node get wrappers
fce8c3a65 CI: Change tinycore URL
8b98dfafd CI: Disable mysql counters for grenade
1cb371327 CI: Try to isolate test failures in neutron vif logic
9c0b4c90a Fix Cinder Integration fallout from CVE-2023-2088
65b8895e8 Update docs: Ironic uses launchpad now
912dcbbdc CI: Mark BFV job non-voting for now
c8c83ef54 Migrate to pysnmp lextudio ecosystem
995e38cd7 Don't return the in-flight SQL handler
9da6dfd73 Fix self_owned_node policy check
17d3b72b3 Remove indicators list by component from api-ref
3139460cd Imported Translations from Zanata
1b8c0be0b Fix api-ref v1-indicators
cec72275a CI: Fix another network test
451857777 CI: Modify dhcp client ID fail
b48dfd44c Use monotonic time for hashring reset
cae05c70e Make rbac enforced test non-voting for the time being
7a5f80cec Fix anaconda stage2_id loading from image properties
03cd9788e Support longer checksums for redfish firmware upgrade
7f281392c Change wholedisk image checksum to sha256
f10958a54 Handle MissingAttributeError when using OOB inspections to fetch MACs
c03a5b44e Remove autocommit, again.
75b881bd3 Fix DB/Lock session handling issues
f2605e928 Remove use of nomodeset by default
3f09bdcf9 [iRMC] Fix parse_driver_info bug enforcing SNMP v3 under FIPS mode
c7b8236ab Configure docs: we no longer use storyboard
510a612ee Add ablity to power off nodes in clean failed
e0c0b771e [iRMC] Fix typo of Python string format in log message
153d4d838 Upgrade to latest hacking - v6
708354573 tests: Replace invalid UUIDs
00aadf570 db: Resolve SAWarning warnings
5268c50a7 Run metal3 integration with ironic-image from source
a5a737e38 Set ironic-grenade to wait 120 seconds
9f422a8df Fix requests calls with timeouts
cc9fa8526 Remove outdated API version information from the enrollment docs
59c6ad96c Always fall back from hard linking to copying files
4bcef0eee Do not log into the template1 database during test-setup
ebc1053ce Exclude all files starting with . from flake8 tests
3e21560bf Remove all references to the "cpus" property
b201e52f8 [CI] Fix port list benchmark
d0448ea20 Use main branch of metal3-dev-env to run metal3 integration job
21437135a Add error logging on lookup failures in the API
6341003da Enables boot modes switching with Anaconda deploy for ilo driver
c5e004a73 Fixes Secureboot with Anaconda deploy
e30ba65f9 Refactoring: clean up inspection data handlers
3dd54a110 Refactoring: DRY in the root API controller
0e7c6f978 Refactoring: create ironic.conductor.inspection
b13ce6eb3 Imported Translations from Zanata
48ea2c5e7 Update master for stable/2023.1
f00da959e Do not recalculate checksum if disk_format is not changed
a6d87a608 Add a non-voting metal3 CI job
bcf6c1226 Clean out agent token even if power is already off
47b590948 Wipe Agent Token when cleaning timeout occcurs
005f21c0d Fix auth_protocol and priv_protocol for SNMP v3
d43f7fbc6 Refactoring: extract some common functions from the inspector code
9acfd5136 Restructure the inspector module in preparation for its expansion
b41d5d514 Document [fake] delay config values
6a9e319fb On rpc service stop, wait for node reservation release


Diffstat (except docs and test files)
-------------------------------------

README.rst                                         |   2 +-
api-ref/source/baremetal-api-v1-indicators.inc     |  45 +---
api-ref/source/baremetal-api-v1-nodes.inc          |  18 ++
api-ref/source/conf.py                             |   2 +-
api-ref/source/parameters.yaml                     |  13 +
.../node-indicators-component-list-response.json   |  45 ----
.../source/samples/node-inventory-response.json    |   1 -
devstack/lib/ironic                                |  17 +-
devstack/tools/ironic/scripts/cirros-partition.sh  |   2 +-
.../install/include/kernel-boot-parameters.inc     |   4 +-
driver-requirements.txt                            |   4 +-
ironic/api/controllers/v1/__init__.py              | 146 +++--------
ironic/api/controllers/v1/node.py                  | 206 +++++++++++----
ironic/api/controllers/v1/ramdisk.py               |   6 +-
ironic/api/controllers/v1/utils.py                 |  17 +-
ironic/api/controllers/v1/versions.py              |   4 +-
ironic/common/cinder.py                            |  71 +++++-
ironic/common/hash_ring.py                         |   4 +-
ironic/common/image_service.py                     |  35 +--
ironic/common/keystone.py                          |  24 +-
ironic/common/kickstart_utils.py                   |   4 +-
ironic/common/molds.py                             |   6 +-
ironic/common/policy.py                            |  12 +-
ironic/common/pxe_utils.py                         |   3 +-
ironic/common/release_mappings.py                  |   4 +-
ironic/common/rpc_service.py                       |  18 +-
ironic/common/utils.py                             |  28 ++
ironic/conductor/base_manager.py                   |  17 +-
ironic/conductor/inspection.py                     | 108 ++++++++
ironic/conductor/manager.py                        | 100 +-------
ironic/conductor/utils.py                          |  17 +-
ironic/conf/conductor.py                           |   8 +
ironic/conf/default.py                             |   5 +-
ironic/conf/ilo.py                                 |   2 +-
ironic/conf/inventory.py                           |  15 +-
ironic/conf/opts.py                                |   1 -
ironic/conf/pxe.py                                 |   2 +-
ironic/conf/redfish.py                             |   2 +-
ironic/db/api.py                                   |  10 -
ironic/db/sqlalchemy/__init__.py                   |  23 +-
.../163040c5513f_add_firmware_information.py       |  50 ++++
.../versions/fe222f476baf_add_parent_node_field.py |  35 +++
ironic/db/sqlalchemy/api.py                        | 269 ++++++++++++--------
ironic/db/sqlalchemy/models.py                     |  21 +-
ironic/drivers/base.py                             |   4 +-
.../ansible/playbooks/library/stream_url.py        |   3 +-
ironic/drivers/modules/deploy_utils.py             |  37 ++-
ironic/drivers/modules/drac/inspect.py             |  14 -
ironic/drivers/modules/drac/raid.py                |   2 +-
ironic/drivers/modules/ilo/boot.py                 |  12 +
ironic/drivers/modules/inspect_utils.py            | 129 +++++-----
ironic/drivers/modules/inspector/__init__.py       |  15 ++
ironic/drivers/modules/inspector/client.py         |  57 +++++
.../{inspector.py => inspector/interface.py}       | 105 ++------
ironic/drivers/modules/irmc/common.py              |  12 +
ironic/drivers/modules/irmc/inspect.py             |   5 +-
ironic/drivers/modules/irmc/management.py          |  18 +-
ironic/drivers/modules/pxe.py                      |  15 +-
ironic/drivers/modules/pxe_base.py                 |   7 +-
ironic/drivers/modules/redfish/firmware_utils.py   |  18 +-
ironic/drivers/modules/redfish/inspect.py          |   4 +-
ironic/drivers/modules/redfish/management.py       |  11 +-
ironic/drivers/modules/snmp.py                     |   4 +-
ironic/hacking/checks.py                           |   2 +-
ironic/objects/node.py                             |   8 +-
.../unit/drivers/modules/drac/test_inspect.py      |  22 --
.../unit/drivers/modules/inspector/__init__.py     |   0
.../unit/drivers/modules/inspector/test_client.py  |  65 +++++
.../test_interface.py}                             |  70 +----
.../unit/drivers/modules/irmc/test_inspect.py      |   6 -
.../unit/drivers/modules/network/test_common.py    |  27 +-
.../drivers/modules/redfish/test_firmware_utils.py |  24 ++
.../unit/drivers/modules/redfish/test_inspect.py   |  18 +-
.../drivers/modules/redfish/test_management.py     |  10 +
.../unit/drivers/modules/test_deploy_utils.py      |  73 +++++-
.../unit/drivers/modules/test_inspect_utils.py     | 136 ++++------
playbooks/metal3-ci/fetch_kube_logs.yaml           |  32 +++
playbooks/metal3-ci/fetch_pod_logs.yaml            |  24 ++
playbooks/metal3-ci/post.yaml                      | 194 ++++++++++++++
playbooks/metal3-ci/run.yaml                       |  39 +++
.../Cleanfail-power-off-13b5fdcc2727866a.yaml      |   8 +
.../add-parent-node-support-10bd42abd008db6f.yaml  |  23 ++
.../notes/bug-2010613-3ab1f32aaa776f28.yaml        |   7 +
.../notes/cinder-2019892-6b5a9de5c5f05aa6.yaml     |  16 ++
.../notes/cross-link-1ffd1a4958f14fd7.yaml         |   5 +
...b-sqlite-OperationalError-7934dbda2a21c69e.yaml |   6 +
...nforcing-snmpv3-with-fips-e45971d363925ec3.yaml |   6 +
...atibility-patch-situation-c246d2b59b2e8a78.yaml |   8 +
.../fix-power-off-token-wipe-e7d605997f00d39d.yaml |   6 +
...ix-self-owned-node-policy-fc2dae357879dc33.yaml |   7 +
...a_deploy_with_ilo_drivers-16637adb62f0ed2f.yaml |   5 +
...boot_with_anaconda_deploy-84d7c1e3bbfa40f2.yaml |   4 +
.../graceful_shutdown_wait-9a62627714b86726.yaml   |  15 ++
...g-ethernetinterfaces-attr-7e52f7259fe66762.yaml |   9 +
releasenotes/notes/no-cpus-c79717303470bf3c.yaml   |   6 +
.../notes/no-recalculate-653e524fd6160e72.yaml     |   5 +
...sh-firmware-sha256-sha512-3e40c3a087fe42b4.yaml |   6 +
...-model-query-from-upgrade-af227b6c8a5d654a.yaml |   9 +
.../notes/remove-nomodset-7a352a9519c1045b.yaml    |  22 ++
...ken-upon-cleaning-timeout-c9add514fad1b02c.yaml |   7 +
releasenotes/source/2023.1.rst                     |   6 +
releasenotes/source/conf.py                        |   2 +-
releasenotes/source/index.rst                      |   1 +
.../locale/en_GB/LC_MESSAGES/releasenotes.po       |  55 +++-
test-requirements.txt                              |   4 +-
tools/benchmark/generate-statistics.py             |   2 +
tools/test-setup.sh                                |   2 +-
tox.ini                                            |   4 +-
zuul.d/ironic-jobs.yaml                            |  13 +
zuul.d/metal3-jobs.yaml                            |  30 +++
zuul.d/project.yaml                                |  13 +-
159 files changed, 3129 insertions(+), 1473 deletions(-)


Requirements updates
--------------------

diff --git a/driver-requirements.txt b/driver-requirements.txt
index 876e817cb..c5a90f2d0 100644
--- a/driver-requirements.txt
+++ b/driver-requirements.txt
@@ -8 +8,3 @@ proliantutils>=2.14.0
-pysnmp>=4.3.0,<5.0.0
+pysnmp-lextudio>=5.0.0 # BSD
+pyasn1-lextudio>=1.1.0 # BSD
+pyasn1-modules-lextudio>=0.2.0 # BSD
diff --git a/test-requirements.txt b/test-requirements.txt
index 0c4bdb0ca..57c4a9c2c 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -16 +16,3 @@ WebTest>=2.0.27 # MIT
-pysnmp>=4.4.12
+pysnmp-lextudio>=5.0.0 # BSD
+pyasn1-lextudio>=1.1.0 # BSD
+pyasn1-modules-lextudio>=0.2.0 # BSD