Open Stack

We are delighted to announce the release of:

ironic 22.1.0: OpenStack Bare Metal Provisioning

This release is part of the bobcat release series.

The source is available from:

    https://opendev.org/openstack/ironic

Download the package from:

    https://tarballs.openstack.org/ironic/

Please report issues through:

    https://storyboard.openstack.org/#!/project/943

For more details, please see below.

22.1.0
^^^^^^


New Features
************

* Adds a capability for syncrhonous steps to be executed through the
  cleaning and deployment steps framework upon child nodes, as
  associated through the "parent_node" field. The new, optional step
  arguments are a boolean value of "execute_on_child_nodes", and
  "limit_child_node_execution" which consists of a list of node UUIDs.
  The ability to invoke this permisison requires the ability to set a
  provision state action upon the child node in the RBAC model.

* Adds a "power_on", "power_on", and "reboot" reserved step name
  actions which toggles power through the conductor. This allows
  embedded devices such as child nodes to have power state toggled as
  part of the parent node's cleaning or deployment sequnece, if so
  stated through the supplied configuration or deployment template.

* Adds a "clean hold" and a "deploy hold" provision state in which
  baremetal nodes can be put in utilizing specialed "hold" cleaning
  and deployment steps. Allowing for patterns and processes where
  Ironic's work is intentionally paused to allow for any external or
  operator processes to take place. In these new states, a "unhold"
  provision state verb can be used to inform Ironic to proceed. The
  "abort" verb is also a possible option should operators wish to
  start over.

* Adds the ability to send an "unhold" provision state verb
  utilizing API version *1.84*.

* Uses Redfish to collect the available hardware inventory
  information and stores it in the right format. Information collected
  includes cpu information including "count", "architecture",
  "model_name", and "frequency", disk "size" (in bytes), interface
  "mac_address", "system_vendor" information including "product_name",
  "serial_number" and "manufacturer", and "current_boot_mode".

* Adds a "wait" clean/deploy step, which takes an optional argument,
  passed in a step definition of "seconds" to force an explicit pause
  of the current process. Otherwise the next heartbeat action triggers
  resumption of the process.

* The "ilo" hardware type firmware upgrade steps, now support
  checksum determination by legnth in order to allow SHA256 and SHA512
  checksums to be supplied by the step caller.

* Methods in vendor interfaces may now be decroated with
  "clean_step" and "deploy_step" decorators.

* The "ipmitool" vendor interface's "send_raw" method can now be
  called as a part of cleaning or deployment steps with an "raw_bytes"
  argument matching how it can be called with the vendor passthru
  interface.


Upgrade Notes
*************

* This release removes two internal foreign key constraints which
  were redundant and which SQLAlchemy indicated may result in an error
  at some point in time. No action is required by an operator for
  this.


Deprecation Notes
*****************

* The use of a SQLite database with mutli-process (i.e. "ironic-api"
  and "ironic-conductor" services) is not supported, and the ability
  to launch a dedicated "ironic-api" process with a SQLite database
  backend will be an error in the future. In this case, the single
  process combined API and Conductor service should be utilized.

* The default value of the "[inspector]require_managed_boot" option
  will change from "False" to "True" in the future, causing in-band
  inspection to fail if the boot interface cannot prepare the ramdisk
  boot (e.g. in case of missing ports). Please set this option to an
  explicit value to avoid the behavior change.


Bug Fixes
*********

* Adds a database write retry decorate for SQLite failures reporting
  "database is locked". By default, through the new configuration
  parameter "[database]sqlite_max_wait_for_retry", retries will be
  performed on failing write operations for up to *30* seconds.

  This value can be tuned, but be warned it is an expotential backoff
  retry model, and HTTP requests can give up if no response is
  received in a reasonable time, thus *30* seconds was deemed a
  reasonable default.

  The retry logic can be disabled using the "[database]sqlite_retries"
  option, which defaults to *True*. Users of other, mutli-threaded
  /concurrent-write database platforms are not impacted by this
  change, as the retry logic recognizes if another database is in use
  and bypasses the retry logic in that case. A similar retry logic
  concept already exists with other databases in the form of a
  "Database Deadlock" retry where two writers conflict on the same row
  or table. The database abstraction layer already handles such
  deadlock conditions. The SQLite file based locking issue is
  unfortunately more common with file based write locking as the
  entire file, in other words the entire database, to perform the
  write operation.

* Fixes issues with locks related to the execution of periodic tasks
  where the task has a lingering transaction. For more information
  please see bug 2027405
  (https://bugs.launchpad.net/ironic/+bug/2027405).

* Fixes a bug that occurs when a node is inspected more than once
  and the database is configured as a storage backend: a new node
  inventory entry is added in the database for each inspection result,
  causing more than one inventory to exist for the node in the
  node_inventory table.

  This is handled by:

  * Deleting any previous inventory entries for a node before adding
    a new entry in the database.

  * Retrieving the most recent node inventory from the database when
    the database is queried. (To cater for databases that already
    contain duplicate node inventories due to the bug.)

* Fixes the bug where provisioning a Redfish managed node fails if
  the BMC only supports virtual media devices limited to MediaType of
  DVD (and not CD). Also ddds handling of BadRequest exceptions while
  iterating through the list of virtual media devices. This fix is
  needed to successfully provision machines such as Cisco UCSB and
  UCSX.

* Fixes the bug where provisioning a Redfish managed node fails if
  changing BIOS settings is attempted on a BMC that doesn't provide
  supportedApplyTime information. This is done by adding handling of
  AttributeError exception in apply_configuration() method.

* Database locks with a "sqlite" database backend should now be
  lessened as the conductor will no longer perform a keepalive
  heartbeat operation when the use of SQLite has been detected.


Other Notes
***********

* Fixes the generated state machine diagram and updates it to match
  the current state of the code.

* The "ipmitool" vendor passthrough interface method no longer
  requires a "http_method" parameter. This is optional in the code
  base, but included on all API initiated vendor passthru method
  calls. The value was not utilized.

Changes in ironic 22.0.0..22.1.0
--------------------------------

9578eb5f7 Don't hold on to the API request dbapi
4ba418716 Retool sqlite retries
e33289816 Retry connecting vmedia through a DVD device if available
0bda65226 Fix missing oslo.versionedobjects library option
f93712d7a Prevent MissingAttribute error when supportedApplyTime missing
23f4a7d99 Support sha256/sha512 with the ilo firmware upgrade logic
3eca0d871 tox: Remove basepython
d2c11df69 Fix typo in deploy_templates docs
d97deb84e Add python3.10 support in testing runtime
bb9b9001a DB: Select upon delete for allocations
cc9af373e DB: Streamline allocation interactions
81b35931d Add a list of children nodes unit test
8fc8372e7 Add wait step
36665a089 Fix ks_template property to be processed only for anaconda deploy
2d8986bda Fix retry logic logging
091edb063 Retry SQLite DB write failures due to locks
e80e029b4 Log when a periodic is completed
0099d1812 Don't actually heartbeat with sqlite
267e61bbc Document caveats of running with SQLite
f4be664a8 Stop splitting installation docs per distros
76c075269 Enable vendor interfaces to be called as steps
1335402f4 Add additional logging on iLO power failure
afada321d Very basic in-band inspection with the "agent" interface
6efa2119e Add the initial skeleton of the agent inspect interface
fb978dab1 DB: Fix result set locking with periodics
8d2e93f30 Add job to test with SQLAlchemy master (2.x)
aecb58108 Firmware Interface
0a11855d3 CI: Use focal dnsmasq
cb60b8a95 Fix db migration tests for sqlalchemy 2.0
6c35a4442 Move standalone jobs to focal
6d3c4ced5 Disable spanning tree
c6610c2f7 Fix several issues in the lock/release database code
e8d6a890f Remove python 3.6 mock hack
c4e3100d5 Add hold steps
f69e9da1d Fix multiple things in CI
5cad8ac77 Unit tests: Isolate mysql test migrations
048bff1da Use tox env variables in coverage tests
847563d2d Correct two mistakes in the /continue_inspection API
642811621 Fix the HTTP code of the BadRequest exception
a600895ba Execute tests by class, not randomly
cf49d54e6 CI: minor fix to irmc driver clean_up_instance testing
c392814ca CI: Fix PXE Ananconda cleanup test
dad672429 Add DB API for Firmware and Object
bc7771bf3 CI: Change migrations timeout to be >60 seoncds
d5931aded Revert "Fix IRONIC_IMAGE_NAME=non-existent-image"
d5e4f013c Skip tests that fail occassionally in CI
402c32094 Handle SAWarning around allocations FK Constratins
46e4f447f Disable WAL Pragma for Unit Testing
4a570042c Fully ensure counts are out of scope of cxtmgr
dd79ae5e2 Ensure all sqla objects descoped before ending txn
3d869bca2 Fix test_migrations with firmware information.
ee963d3f0 Revert "Disabling test_upgrade_twice temporarily for CI fix"
9be84608f Fix SQLAlchemy engine connection listener
2f8ee2cf4 Fix IRONIC_IMAGE_NAME=non-existent-image
dfb7f05ff Allow setting migrations timeout value from tox
f052bde94 Add test timeout to tox config
b4f8209b9 Disabling test_upgrade_twice temporarily for CI fix
ca5b2feee Mock sleep in unit tests that rely on it
fa2d6685f Handle duplicate node inventory entries per node
0370f5ac9 Migrate the inspector's /continue API
f43464329 Use jammy for base jobs
c0643e9d0 Be explicit about bugfix branches support lifetime
b3d7ba88d Add to Redfish hardware inventory collection
013ac0cb4 execute on child node support
58388212b Prepare [inspector]require_managed_boot to change to True in the future


Diffstat (except docs and test files)
-------------------------------------

.../source/baremetal-api-v1-deploy-templates.inc   |   2 +-
api-ref/source/baremetal-api-v1-nodes-firmware.inc |  48 ++
.../node-firmware-components-list-response.json    |  20 +
devstack/lib/ironic                                |  12 +
devstack/plugin.sh                                 |   1 +
devstack/tools/ironic/scripts/setup-network.sh     |   2 +-
.../install/include/common-prerequisites.inc       |  53 ++-
.../install/include/configure-ironic-api.inc       |   4 +-
.../install/include/configure-ironic-conductor.inc |   4 +-
.../include/configure-ironic-singleprocess.inc     |   8 +-
ironic/api/config.py                               |   1 +
ironic/api/controllers/v1/__init__.py              |   6 +
ironic/api/controllers/v1/driver.py                |   4 +
ironic/api/controllers/v1/firmware.py              |  75 +++
ironic/api/controllers/v1/node.py                  |  50 +-
ironic/api/controllers/v1/ramdisk.py               | 145 +++++-
ironic/api/controllers/v1/utils.py                 |  31 +-
ironic/api/controllers/v1/versions.py              |   7 +-
ironic/api/hooks.py                                |  11 +-
ironic/cmd/api.py                                  |  12 +
ironic/cmd/conductor.py                            |  36 ++
ironic/common/exception.py                         |  25 +-
ironic/common/hash_ring.py                         |   7 +-
ironic/common/policy.py                            |  17 +-
ironic/common/pxe_utils.py                         |  53 ++-
ironic/common/release_mappings.py                  |   7 +-
ironic/common/states.py                            |  28 ++
ironic/common/utils.py                             |  14 +
ironic/conductor/base_manager.py                   |   5 +
ironic/conductor/cleaning.py                       |  93 +++-
ironic/conductor/deployments.py                    |  92 +++-
ironic/conductor/inspection.py                     |  44 ++
ironic/conductor/manager.py                        |  64 ++-
ironic/conductor/periodics.py                      |   1 +
ironic/conductor/rpcapi.py                         |  18 +-
ironic/conductor/steps.py                          |  80 ++++
ironic/conductor/utils.py                          |   7 +-
ironic/conf/conductor.py                           |  25 +-
ironic/conf/database.py                            |  13 +-
ironic/conf/default.py                             |   5 +
ironic/conf/fake.py                                |   6 +
ironic/conf/inspector.py                           |  38 +-
ironic/db/api.py                                   |  61 +++
ironic/db/sqlalchemy/__init__.py                   |  13 +-
.../d163df1bab88_remove_extra_fk_constraint_in_.py |  50 ++
.../versions/dd67b91a1981_add_allocations_table.py |  14 +-
ironic/db/sqlalchemy/api.py                        | 450 ++++++++++++------
ironic/db/sqlalchemy/models.py                     |   7 +-
ironic/drivers/base.py                             |  72 ++-
ironic/drivers/fake_hardware.py                    |   5 +
ironic/drivers/generic.py                          |   7 +-
ironic/drivers/hardware_type.py                    |   5 +
ironic/drivers/irmc.py                             |   4 +-
ironic/drivers/modules/agent_base.py               |   9 +-
ironic/drivers/modules/fake.py                     |  34 ++
ironic/drivers/modules/ilo/common.py               |  22 +-
ironic/drivers/modules/ilo/power.py                |   7 +-
ironic/drivers/modules/inspect_utils.py            | 171 +++++++
ironic/drivers/modules/inspector/__init__.py       |   3 +-
ironic/drivers/modules/inspector/agent.py          |  83 ++++
ironic/drivers/modules/inspector/interface.py      | 141 ++++--
ironic/drivers/modules/inspector/nics.py           | 209 +++++++++
ironic/drivers/modules/ipmitool.py                 |  31 +-
ironic/drivers/modules/noop.py                     |  10 +
ironic/drivers/modules/redfish/bios.py             |  16 +-
ironic/drivers/modules/redfish/boot.py             |  41 +-
ironic/drivers/modules/redfish/inspect.py          |  97 +++-
ironic/drivers/redfish.py                          |   5 +-
ironic/objects/__init__.py                         |   1 +
ironic/objects/firmware.py                         | 161 +++++++
ironic/objects/node.py                             |  10 +-
.../unit/drivers/modules/inspector/test_agent.py   | 128 ++++++
.../unit/drivers/modules/inspector/test_nics.py    | 311 +++++++++++++
.../unit/drivers/modules/network/test_common.py    |   6 +
.../unit/drivers/modules/redfish/test_bios.py      |  21 +
.../unit/drivers/modules/redfish/test_boot.py      |  47 ++
.../unit/drivers/modules/redfish/test_inspect.py   | 193 +++++++-
.../unit/drivers/modules/test_agent_client.py      |   4 +-
.../unit/drivers/modules/test_inspect_utils.py     | 211 ++++++++-
...add-execute-on-child-node-20910aecb8f8b714.yaml |  16 +
.../notes/add-hold-states-7be5804d6f3a119a.yaml    |  18 +
...edfish-hardware-inventory-830ebc0b2e0f50c2.yaml |   8 +
.../add-sqlite-db-retries-f493d5d7aa6db78b.yaml    |  25 +
.../notes/add-wait-step-3751e7918afdd199.yaml      |   7 +
...lite-locks-with-periodics-362de3c63bc23e4b.yaml |   6 +
...uplicate-node-inventories-fc41a7decbb6da76.yaml |  15 +
.../handle-dvd-only-vmedia-f4971a013a8aafd0.yaml   |   8 +
...-supportedapplytimes-attr-fbacc7ca3c399e83.yaml |   7 +
...de-supports-sha256-sha512-ae76569042750a07.yaml |   6 +
...the-conductor-with-sqlite-7cfc71e717d69bc5.yaml |  13 +
...allocation-fk-constraints-0f59170f4d164a6e.yaml |   6 +
.../require-managed-boot-7d8965d4ac5fbd21.yaml     |   8 +
...-interface-step-decorated-a673f608c5f5721a.yaml |  15 +
requirements.txt                                   |   2 +-
setup.cfg                                          |   6 +
tools/config/ironic-config-generator.conf          |   1 +
tools/states_to_dot.py                             |   2 +-
tox.ini                                            |  22 +-
zuul.d/ironic-jobs.yaml                            |  33 +-
zuul.d/project.yaml                                |   4 +
154 files changed, 5790 insertions(+), 1105 deletions(-)


Requirements updates
--------------------

diff --git a/requirements.txt b/requirements.txt
index 2f4813baa..d6918b9fe 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -43 +43 @@ oslo.messaging>=14.1.0 # Apache-2.0
-tenacity>=6.2.0  # Apache-2.0
+tenacity>=6.3.1  # Apache-2.0