We high-spiritedly announce the release of: ironic-inspector 5.0.0: Hardware introspection for OpenStack Bare Metal This release is part of the ocata release series. The source is available from: http://git.openstack.org/cgit/openstack/ironic-inspector Download the package from: https://tarballs.openstack.org/ironic-inspector/ Please report issues through launchpad: http://bugs.launchpad.net/ironic-inspector For more details, please see below. 5.0.0 ^^^^^ New Features ************ * Extend the introspection status returned from "GET@/v1/introspection/<Node Id>" to contain the "uuid", "started_at" and "finished_at" fields. * Add a plugin to parse raw LLDP Basic Management, 802.1, and 802.3 TLVs and store the data in Swift. * Add an API endpoint for listing introspection statuses. Operators can use this to get the status for all running or previously run introspection processing. * Introduce a new configuration option "api_max_limit" that defines the maximum number of items per page when API results are paginated. * InfiniBand interface discovery is now supported through introspection. The ironic-inspector will add the client-id to the corresponding ironic port that represents the InfiniBand interface. The ironic-inspector should be configured with a list of interfaces "firewall.ethoib_interfaces" to indicate which Ethernet Over InfiniBand Interfaces are used for DHCP. * Node introspection state is now kept in a dedicated database column. The introspection is now using a finite state machine. The state isn't exposed to the user yet. * Adds support for using operators with the root device hints mechanism. The supported operators are "=", "==", "!=", ">=", "<=", ">", "<", "s==", "s!=", "s>=", "s>", "s<=", "s<", "<in>", "<all-in>" and "<or>". * Looking up nodes during introspection or discovery now supports multiple attributes matching. For example, two nodes can use the same "bmc_address" and still can be distinguished by MAC addresses. * Avoid failing introspection on diskless nodes. The node property "local_gb == 0" is set in that case. Known Issues ************ * Due to the nature of the NodeInfo.state attribute (being updated independently from the rest of the node_info attributes) if a (DB) connection was lost before the Node.state column was updated, Node.finished_at and Node.error columns may not be in sync with the Node.state column. Upgrade Notes ************* * Add a new dependency, "pytz". * A database migration is required to change some columns from Float to DateTime type. This may take some time based on the number of introspection statuses in DB. * Removed previously deprecated authentication options from "ironic", "swift", and "keystone_authtoken" sections. * Removed long deprecated support for "discoverd" section in configuration file. * The default value for the configuration option "introspection_delay_drivers" was changed to ".*", which means that by default "introspection_delay" is now applied to all drivers. Set "introspection_delay" to 0 to disable the delay. * Node.state and Node.version_id database columns are introduced. * The introspection state column defaults to the state "finished" unless the introspection error column value on a node row isn't null, then node state is set to "error". * Uniqueness of a node "bmc_address" isn't enforced any more. * The primary key of the "attributes" table is relaxed from the "attributes.name, attributes.value" column pair to a new column "attributes.uuid". Deprecation Notes ***************** * The configuration option "log_bmc_address" is deprecated. * Support for setting IPMI credentials via ironic-inspector is deprecated and will be removed completely in Pike. A new API version 1.9 was introduced with this feature de-activated. For reasoning see https://bugs.launchpad.net/ironic-python-agent/+bug/1654318. * The configuration option "introspection_delay_drivers" is deprecated. Bug Fixes ********* * Change database columns "started_at" and "finished_at" to type DateTime from type Float so that timestamps fit into these columns correctly. * Fix bug where periodic clean up failed with DBDeadlock if introspection timed out. * Ensure the configuration options "firewall.firewall_update_period" and "clean_up_period" are applied to the "periodic_clean_up" and "periodic_update" tasks after the config file is read. * LLC hook now formats the chassis ID and port ID MAC addresses into Unix format as expected by ironic. * LLC hook ensures that correct port information is passed to the patch_port function * LLC hook no longer assumes all inspected ports are added to ironic * Loopback BMC addresses (useful e.g. with virtualbmc) are no longer used for lookup. * Introspection fails on nodes with the same IPMI address but different IPMI ports. Other Notes *********** * Default API version is temporary pinned to 1.8 (before deprecating setting IPMI credentials). It will be reset to the latest version again when support for setting IPMI credentials is removed. Changes in ironic-inspector 4.2.0..5.0.0 ---------------------------------------- 81cef3d Fix some grammar and spelling issues in release notes 0ce5cdb Find a node by multiple attributes 7e8cdc0 Pass session directly to swiftclient 5d0c739 Ensure devstack does not try to delete a physical NIC 02113d2 Switch to use test_utils.call_until_true 1101741 Add node_info to some messages and clean up docstrings for lldp plugin 500623f Fix updating rows in d00d6e3f38c4 migration c2c27bb Typo fix: infromation => information ffcb93e Prepare for using standard python tests 25015da Trivial: improve logging in FSM transition 1dce3b1 Adding InfiniBand Support 8834927 Add plugin to process basic management LLDP TLVs 57ee71b Clean up logging related to new state machine d6c3ca1 Update external links in the documentation 73584d2 Clean up deprecated configuration options aeb72b8 Switch to decorators.idempotent_id 43fe364 Updated from global requirements 6e2ea62 Allow diskless nodes introspection 9a5cc11 Remove heading "Team and repository tags" 0a32123 Deprecate introspection_delay_drivers option and make it no-op d429cd7 [devstack] Stop installing jq, we're not using it in tests any more 2de5419 Deprecated log_bmc_address option c4cbfa7 [devstack] Remove bridge only if it's not OVB_PHYSICAL_BRIDGE 71099c9 Change (started|finished)_at column type e0f8b23 Make grenade actually upgrade our source code b95c4c1 Devstack: don't rely on Ironic local vars 31cd5f4 Fix DBDeadlock during clean up db2402b Skip brbm-inspector veth if IRONIC_IS_HARDWARE == True 75a99f4 Updated from global requirements 59a2009 Updated from global requirements 635db52 Deprecate setting IPMI credentials 51015b1 Do not use loopback BMC addresses for lookup 5a415e2 Updated from global requirements e28b0b2 Functional test node remove 436c3c7 Updated from global requirements 444c01b Updated from global requirements 7456513 Add troubleshooting step for virtual box 46f4019 Updated from global requirements 809fd43 Remove unused "service" argument from tempest client manager d76e845 Tempest test tag baremetal doesn't exist 2230cf9 Updated from global requirements 8916396 Use the device hints matching mechanism from ironic-lib 01b8359 Fix test when running with SQLite 3.7.17 from CentOS 7 2080881 Add drac_address to ipmi_address_fields fe7b4e3 Update config sample ecf898b Revert "Use liberty-eol tag for liberty release notes" f6d02e1 Use oslo_serialization.base64 to follow OpenStack Python3 df0a1b9 Updated from global requirements 3ddc061 Introducing node introspection state management 9af663b Use liberty-eol tag for liberty release notes b917c74 Add author and author-email in setup.cfg bcfc86e Remove upgrade from non-ironic setup c106d91 Remove default_params_with_timeout_values from tempest client 7e3ff66 Updated from global requirements 2bb48c6 Document upgrade procedure 473882e Clean up retained code for grenade workaround 715f4e2 Show team and repo badges on README 5de0c24 Do not source old/localrc twise in grenade 253fd37 Don't recreate db during grenade upgrade 185effb Use function is_valid_mac from oslo.utils 7cb40d5 Add API for listing all introspection statuses 25c5a77 Stop disabling the Neutron tempest plugin 3cdaf0d Fix passing incorrect args to logging 59cf107 Remove the generic eventlet monkey patch from test a42793b Test discovered nodes are in ENROLL state and fix typo e7b54b3 Fixed incorrect string type checking 2d228cd Add !requiretty to sudoers config in install document 4ba285b Add py35 to tox job list ba9adaa Update to hacking 0.12.0 and use new checks cd253c3 Updated from global requirements f52ee7a Add a test for introspection abort action verification. c30b77f Updated from global requirements 2401f0c Allow to configure cache clean up period cb8db15 Switch func3 environment to Python 3.5 739e9c2 Updated from global requirements 3928c7d Move assertStatus to Base functional test and rename it for clarity 2863c84 Updated from global requirements 3b15527 UUID, started_at, finished_at in the status API 4eb7786 Only disable Neutron tests when our Grenade is running 75ab592 Bump hacking to 0.11.0 in test-requirements a372bec Updated from global requirements c4ec98b Correct conf.py missing space 118f149 Fix a typo in base.py ad24055 Stop adding ServiceAvailable group option 1d2b617 Enable release notes translation 89aeb27 Updated from global requirements cac5ab1 devstack: remove old exercise scripts 09b3e94 Updated from global requirements c18d3cf LLC Hook: Do not assume interfaces are added to Ironic 20d941e LLC Hook: Fix patching Ironic ports 22cb815 TrivialFix: Fix typo in the configuration file cc931f5 Updated from global requirements 3332709 TrivialFix: Remove default=None when set value in Config 8da54e5 Use assertEqual() instead of assertDictEqual() 937b4d7 Update reno for Newton 9efe03f Updated from global requirements Diffstat (except docs and test files) ------------------------------------- CONTRIBUTING.rst | 12 +- README.rst | 9 +- devstack/discovery_exercise.sh | 84 --- devstack/exercise.sh | 238 --------- devstack/exercise_common.sh | 312 ----------- devstack/plugin.sh | 30 +- devstack/upgrade/settings | 2 +- devstack/upgrade/upgrade.sh | 40 +- example.conf | 209 ++++---- ironic_inspector/api_tools.py | 83 +++ ironic_inspector/common/ironic.py | 85 +-- ironic_inspector/common/keystone.py | 77 +-- ironic_inspector/common/lldp_parsers.py | 364 +++++++++++++ ironic_inspector/common/lldp_tlvs.py | 366 +++++++++++++ ironic_inspector/common/swift.py | 65 +-- ironic_inspector/conf.py | 93 ++-- ironic_inspector/db.py | 33 +- ironic_inspector/firewall.py | 60 ++- ironic_inspector/introspect.py | 14 +- ironic_inspector/introspection_state.py | 144 +++++ ironic_inspector/main.py | 68 ++- .../882b2d84cb1b_attribute_constraints_relaxing.py | 90 ++++ ...d00d6e3f38c4_change_created_finished_at_type.py | 69 +++ ...2e48801c8ef_introducing_node_state_attribute.py | 49 ++ ironic_inspector/node_cache.py | 450 +++++++++++++--- ironic_inspector/plugins/base.py | 2 +- ironic_inspector/plugins/lldp_basic.py | 88 ++++ ironic_inspector/plugins/local_link_connection.py | 24 +- ironic_inspector/plugins/standard.py | 69 ++- ironic_inspector/process.py | 89 ++-- ironic_inspector/test/__init__.py | 15 - ironic_inspector/test/base.py | 63 ++- ironic_inspector/test/functional.py | 347 ++++++++++++- .../test/inspector_tempest_plugin/config.py | 15 +- .../test/inspector_tempest_plugin/plugin.py | 18 +- .../services/introspection_client.py | 16 +- ironic_inspector/test/unit/test_api_tools.py | 136 +++++ ironic_inspector/test/unit/test_common_ironic.py | 6 + ironic_inspector/test/unit/test_firewall.py | 111 +++- ironic_inspector/test/unit/test_introspect.py | 187 +++---- ironic_inspector/test/unit/test_keystone.py | 54 +- ironic_inspector/test/unit/test_main.py | 99 +++- ironic_inspector/test/unit/test_migrations.py | 125 +++++ ironic_inspector/test/unit/test_node_cache.py | 577 ++++++++++++++++++--- .../test/unit/test_plugins_lldp_basic.py | 329 ++++++++++++ .../unit/test_plugins_local_link_connection.py | 17 +- .../test/unit/test_plugins_pci_devices.py | 6 +- .../test/unit/test_plugins_standard.py | 31 +- ironic_inspector/test/unit/test_process.py | 113 ++-- ironic_inspector/test/unit/test_swift.py | 18 +- ironic_inspector/test/unit/test_utils.py | 60 +-- ironic_inspector/utils.py | 90 ++-- ...shed_at-in-the-status-API-7860312102923938.yaml | 9 + .../add-lldp-basic-plugin-98aebcf43e60931b.yaml | 4 + ...ll-introspection-statuses-2a3d4379c3854894.yaml | 10 + .../bmc-logging-deprecation-4ca046a64fac6f11.yaml | 4 + ...ished_at_type_to_datetime-c5617e598350970c.yaml | 11 + ...recate-setting-ipmi-creds-1581ddc63b273811.yaml | 12 + ...ted-options-removal-ocata-a44dadf3bcf8d6fc.yaml | 8 + ...x-deadlock-during-cleanup-bcb6b517ef299791.yaml | 5 + ...iodic-tasks-configuration-edd167f0146e60b5.yaml | 6 + .../notes/fix_llc_hook_bugs-efeea008c2f792eb.yaml | 6 + .../fix_llc_port_assume-4ea47d26501bddc3.yaml | 3 + .../notes/infiniband-support-960d6846e326dec4.yaml | 8 + ...delay-drivers-deprecation-1d0c25b112fbd4da.yaml | 10 + .../introspection-state-03538fac198882b6.yaml | 16 + .../notes/ironic-lib-hints-20412a1c7fa796e0.yaml | 6 + .../notes/loopback-bmc-e60d64fe74bdf142.yaml | 5 + ...ultiattribute_node_lookup-17e219ba8d3e5eb0.yaml | 17 + .../notes/optional-root-disk-9b972f504b2e6262.yaml | 5 + releasenotes/source/conf.py | 3 + releasenotes/source/index.rst | 3 +- releasenotes/source/newton.rst | 6 + requirements.txt | 27 +- rootwrap.conf | 2 +- setup.cfg | 3 + test-requirements.txt | 10 +- tools/test-setup.sh | 57 ++ tox.ini | 8 +- 88 files changed, 4510 insertions(+), 1807 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 2808e32..938790f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4 +4,2 @@ -alembic>=0.8.4 # MIT +automaton>=0.5.0 # Apache-2.0 +alembic>=0.8.10 # MIT @@ -5,0 +7 @@ Babel>=2.3.4 # BSD +construct>=2.8.10 # MIT @@ -8,0 +11 @@ futurist!=0.15.0,>=0.11.0 # Apache-2.0 +ironic-lib>=2.5.0 # Apache-2.0 @@ -11,2 +14,2 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT -keystoneauth1>=2.10.0 # Apache-2.0 -keystonemiddleware!=4.1.0,!=4.5.0,>=4.0.0 # Apache-2.0 +keystoneauth1>=2.18.0 # Apache-2.0 +keystonemiddleware>=4.12.0 # Apache-2.0 @@ -14,3 +17,4 @@ netaddr!=0.7.16,>=0.7.13 # BSD -pbr>=1.6 # Apache-2.0 -python-ironicclient>=1.6.0 # Apache-2.0 -python-swiftclient>=2.2.0 # Apache-2.0 +pbr>=1.8 # Apache-2.0 +python-ironicclient>=1.11.0 # Apache-2.0 +python-swiftclient>=3.2.0 # Apache-2.0 +pytz>=2013.6 # MIT @@ -18,2 +22,2 @@ oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.config>=3.14.0 # Apache-2.0 -oslo.db!=4.13.1,!=4.13.2,>=4.10.0 # Apache-2.0 +oslo.config!=3.18.0,>=3.14.0 # Apache-2.0 +oslo.db>=4.15.0 # Apache-2.0 @@ -21 +25 @@ oslo.i18n>=2.1.0 # Apache-2.0 -oslo.log>=1.14.0 # Apache-2.0 +oslo.log>=3.11.0 # Apache-2.0 @@ -24 +28,2 @@ oslo.rootwrap>=5.0.0 # Apache-2.0 -oslo.utils>=3.16.0 # Apache-2.0 +oslo.serialization>=1.10.0 # Apache-2.0 +oslo.utils>=3.18.0 # Apache-2.0 @@ -26 +31 @@ six>=1.9.0 # MIT -stevedore>=1.16.0 # Apache-2.0 +stevedore>=1.17.1 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index fad950e..69a1b29 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -4 +4 @@ -coverage>=3.6 # Apache-2.0 +coverage>=4.0 # Apache-2.0 @@ -6 +6 @@ doc8 # Apache-2.0 -hacking<0.11,>=0.10.0 +hacking<0.13,>=0.12.0 # Apache-2.0 @@ -8,3 +8,3 @@ mock>=2.0 # BSD -sphinx!=1.3b1,<1.3,>=1.2.1 # BSD -oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0 -reno>=1.8.0 # Apache2 +sphinx!=1.3b1,<1.4,>=1.2.1 # BSD +oslosphinx>=4.7.0 # Apache-2.0 +reno>=1.8.0 # Apache-2.0