Open Stack

We are stoked to announce the release of:

ironic-python-agent 6.5.0: Ironic Python Agent Ramdisk

This release is part of the wallaby release series.

The source is available from:

    https://opendev.org/openstack/ironic-python-agent

Download the package from:

    https://tarballs.openstack.org/ironic-python-agent/

Please report issues through:

    https://storyboard.openstack.org/#!/project/openstack/ironic-
python-agent

For more details, please see below.

6.5.0
^^^^^


New Features
************

* Adds UUID of the disks to the inventory of block devices that is
  collected during inspection.

* Adds the ability to bring up VLAN interfaces and include them in
  the introspection report.  A new kernel params field is added -
  "ipa- enable-vlan-interfaces", which defines either the VLAN
  interface to enable, the interface to use, or 'all' - which
  indicates all interfaces.  If the particular VLAN is not provided,
  IPA will use the LLDP information for the interface to determine
  which VLANs should be enabled. See story 2008298
  (https://storyboard.openstack.org/#!/story/2008298).

* Adds a clean step to erase the Linux kernel's pstore. The step is
  disabled by default.

* Adds an configuration option which can be encoded into the ramdisk
  itself or the PXE parameters being provided to instruct the agent to
  ignore bootloader installation or configuration failures. This
  functionality is useful to work around well-intentioned hardware
  which is auto-populating all possible device into the UEFI nvram
  firmware in order to try and help ensure the machine boots. Except,
  this can also mean any explict configuration attempt will fail.
  Operators needing this bypass can use the "ipa-ignore-bootloader-
  failure" configuration option on the PXE command line or utilize the
  "ignore_bootloader_failure" option for the Ramdisk configuration. In
  a future version of ironic, this setting may be able to be overriden
  by ironic node level configuration.

* Deployers in highly-secure environments can now manually set
  Ironic API version instead of relying on unauthenticated
  autodetection via the "ipa-ironic-api-version" on the kernel command
  line. This is not a recommended configuration.

* For Software RAID, the IPA will use partition LABEL along with
  UUID and PARTUUID passed from the conductor to identify the root
  partition. The root file system LABEL can be set as value of the
  "rootfs_uuid" image metadata property.


Security Issues
***************

* If enabled, the new clean step 'erase_pstore' removes all pstore
  entries (the oops/panic logs from a failing kernel) upon cleaning.
  This is to reduce the risk that potentially sensitive data is
  preserved across instantiations (and therefore different users) of a
  bare metal node.


Bug Fixes
*********

* Fixes an issue where intermittent or transitory connection issues
  can cause inspection to fail. The ramdisk now retries to report to
  inspector a total of five times.

* The system file system configuration file for Linux machines, the
  "/etc/fstab" file is now updated to include a reference to the EFI
  partition in the case of a partition image base deployment. Without
  this reference, images deployed using partition images could end up
  in situations where upgrading the bootloader could fail.

* Automatically generated TLS certificates now have their validity
  starting in the past (1 hour by default) to allow for clock skew.

* Fixes the agent process for determining what partition label type
  to utilize when writing partition images. In many cases, this could
  fallback to "msdos" if the instance flavor was not properly labeled.

* Fixes issue where the running system operating mode was not taken
  into account when writing partition images. The agent now utilises a
  helper instead of explicitly expecting the flavor derived
  information to supply all deployment context.

* Fixes an issue where deployments of Fedora or Centos can hang when
  using grub2 with the execution of the "grub2-mkconfig" command not
  returning before the deployment process times out. This is because
  "grub2-mkconfig" triggers "os-prober" which can take an extended
  period of time to evaluate additional unrelated devices for dual-
  boot scenarios. Since operators are not dual booting their machines
  enrolled in ironic, it seems like an un-necessary scan and has thus
  been disabled.

* Correctly decodes error messages from ironic API.

* The "mdadm" utility is no longer a hard requirement. It's still
  required if software RAID is used (even when not managed by ironic).

* Fixes the "write_image" deploy step to actually check and return
  any errors during its execution.

* Fixes the agent's EFI boot handling such that EFI assets from a
  partition image are preserved and used instead of overridden. This
  should permit operators to use Secure Boot with partition images IF
  the assets are already present in the partition image.

* Upon the creation of Software RAID devices, component devices are
  sometimes kicked out immediately (for no apparent reason). This fix
  re-adds devices in such cases in order to prevent the component to
  be missing next time the device is assembled, which, for instance
  may prevent the UEFI ESPs to be installed properly.

* Avoids a traceback when using "install_bootloader" with whole disk
  images. If the root UUID cannot be detected, don't try to call grub.


Other Notes
***********

* Agent configuration files found on attached virtual media or
  config drive devices are now copied to the ramdisk and loaded on
  start up.

Changes in ironic-python-agent 6.4.0..6.5.0
-------------------------------------------

4fb8163 Fix boot mode detection for partition images
246e0cf Change default ironic_lib invocation to flag local booting
a12a574 Add fstab pointer to EFI partition
d69f12e Handle situation when a configdrive is already mounted
78b356c Remove lower-constraints job
88621e1 Avoid a full install in tox environments that do not need it
f9870d5 Prevent broken partition image UEFI deploys
cb6c005 Fix default disk label with partition images
67ee667 Upgrade version of doc8
557293c Generate TLS certificates with validity time in the past
7a83773 Option to enable bootloader config failure bypass
c327735 Fix lower-constraints with the new pip resolver
53dbc87 Correctly decode error messages from ironic API
b9b67fa Copy any configuration from the virtual media
ab8dee0 Make mdadm a soft requirement
6e3f28d Bring up VLAN interfaces and include in introspection report
60900d4 Reuse the docs deps to benefit from constraints
92e26b0 Add clean step 'erase_pstore'
3761a44 Fix vendor info retrieval for some versions of lshw
19c1a73 Remove the unused coding style modules
c7858d3 Add UUID to BlockDevice object
c585603 Log configuration options on start-up
448ded4 Fix physical memory calculation with new lshw
35d412e Updated Implementation of string interpolation delay on LOG messages
694ea74 Support using LABEL as identifier for rootfs
24a4b13 Use TOX_CONSTRAINTS_FILE
3a46586 Add example for custom disk erasure
066a96a Follow-up to API version setting
c3ab4a7 Remove nodeset option
1f15a10 Run dib ipa src jobs on ubuntu focal
a67807b Mark standalone job non-voting/remove from gate
8057556 Allow manual setting of Ironic API Version
71b6abc update lower-constraints.txt
6542a9c Don't run os-prober from grub2-mkconfig
7bcddee Set safe version of hacking
c7f6baf [trivial] Remove redundant list conversion
420ebc0 Do not silently swallow errors in the write_image deploy step
62672de Reduce the duration of retries in the inspector tests
1a67ddd Log a warning of target_boot_mode does not match current boot mode
fc4e0ee Don't try to call GRUB when root UUID is not provided
5c99f60 Use focal for tinyipa src jobs used by ipa-builder
13de98b Use bionic nodeset for DIB centos src jobs
253b488 Software RAID: Re-add missing devices
3ddca46 Add Python3 wallaby unit tests
fb45e58 Update master for stable/victoria
bb27bad Add basic retries for inspection


Diffstat (except docs and test files)
-------------------------------------

examples/README.rst                                |   9 +
examples/custom-disk-erase/example_disk_eraser.py  |  59 ++
examples/custom-disk-erase/setup.cfg               |  20 +
examples/custom-disk-erase/setup.py                |   6 +
ironic_python_agent/api/app.py                     |   2 -
ironic_python_agent/cmd/agent.py                   |  13 +
ironic_python_agent/config.py                      |  36 +-
ironic_python_agent/extensions/base.py             |  11 +
ironic_python_agent/extensions/image.py            | 398 +++++++--
ironic_python_agent/extensions/standby.py          |  11 +-
ironic_python_agent/hardware.py                    | 268 ++++--
ironic_python_agent/inspector.py                   |  20 +-
ironic_python_agent/ironic_api_client.py           |  67 +-
ironic_python_agent/netutils.py                    | 117 +++
ironic_python_agent/tls_utils.py                   |  18 +-
ironic_python_agent/utils.py                       | 125 ++-
lower-constraints.txt                              |  93 --
.../add-block-device-uuid-c8b38264e1688110.yaml    |   5 +
.../add-inspection-retry-1d385f69607c1452.yaml     |   6 +
.../add-vlan-interfaces-cdfeb39d0f3d444d.yaml      |  12 +
.../notes/add_erase_pstore-b109c58ed8f5d351.yaml   |  11 +
...all-failure-to-be-ignored-b99667b13afa9759.yaml |  15 +
...nd-efi-partition-to-fstab-e9f945a4dd19bd7a.yaml |   8 +
.../notes/clock-skew-1fbf542b193cec17.yaml         |   5 +
...n-of-partition-table-type-3c78bf78266e8cef.yaml |   6 +
...mode-for-partition-images-f96cf2b3c27b6533.yaml |   7 +
...edora-grub2-mkconfig-hang-fe22cde231994044.yaml |  11 +
.../notes/ironic-error-97e76d9ddacff039.yaml       |   4 +
...figure-ironic-api-version-517afd0a423036ad.yaml |   7 +
releasenotes/notes/mdadm-d5b8c186182620b1.yaml     |   5 +
.../notes/prepare-image-49744276cef719d5.yaml      |   5 +
...serve-efi-folder-contents-ea1e278b3093ec55.yaml |   7 +
.../notes/readd_missing_devs-2ed85805388b6e42.yaml |   8 +
...-use-label-as-rootfs-uuid-d9a3827180f1a238.yaml |   6 +
.../notes/vmedia-copy-6a58f3183b166c42.yaml        |   5 +
.../notes/whole-disk-grub-0b1b8b9c44e31d28.yaml    |   5 +
releasenotes/source/index.rst                      |   1 +
releasenotes/source/victoria.rst                   |   6 +
test-requirements.txt                              |   6 +-
tox.ini                                            |  25 +-
zuul.d/ironic-python-agent-jobs.yaml               |  21 +-
zuul.d/project.yaml                                |  14 +-
53 files changed, 2842 insertions(+), 464 deletions(-)


Requirements updates
--------------------

diff --git a/test-requirements.txt b/test-requirements.txt
index d07515f..339b33c 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -4 +4 @@
-hacking>=3.1.0,<3.2.0 # Apache-2.0
+
@@ -9,2 +8,0 @@ stestr>=1.0.0 # Apache-2.0
-bashate>=0.5.1 # Apache-2.0
-flake8-import-order>=0.17.1 # LGPLv3
@@ -13,2 +10,0 @@ bandit!=1.6.0,>=1.1.0,<2.0.0 # Apache-2.0
-# Doc test requirements
-doc8>=0.6.0 # Apache-2.0