We contentedly announce the release of: tripleo-common 11.5.0: A common library for TripleO workflows. This release is part of the train stable release series. The source is available from: https://opendev.org/openstack/tripleo-common Download the package from: https://tarballs.openstack.org/tripleo-common/ Please report issues through: https://bugs.launchpad.net/tripleo-common/+bugs For more details, please see below. 11.5.0 ^^^^^^ New Features ************ * Added *modify_only_with_source* to the ContainerImagePrepare set that can be used to limit modify container images to a specific image_source as defined in the services to container images mapping. * Introduce new HEALTHCHECK_DEBUG variable in order to toggle verbosity, defaults to 0 (no verbosity). Setting it to 1 will activate -x flag, among other things. * This patch moves away from "ss" execs, using lsof instead. This allows to drop most of the piping and subshells, making things stronger. * Introduce new HEALTHCHECK_DEBUG variable in order to toggle verbosity, defaults to 0 (no verbosity). Setting it to 1 will activate -x flag, among other things. * Push some verbose output to a third descriptor, visible only if we set the healthcheck to debug. * Removed support for troubleshooting network issues using Skydive (http://skydive.network). Bug Fixes ********* * When the default tag doesn't exist in the container repo during container image prepare, and a tag wasn't set in the actual input for ContainerImagePrepare, the latest tag from the repo will be used instead of failing with a not found error. * Fix bug 1887692 so limit_hosts will take precedence over the blacklisted_hostnames. And therefore Ansible won't be run with two --limit if both limit hosts and blacklisted hostnames are in use. When we want to run Ansible on specific hosts, we will ignore the blacklisted nodes and assume we know what we do. In the case of the scale-down scenario, the unreachable nodes are ignored. * The qemu user on the host gets created using uid/gid 107. Certificates on the host, but also the vhost-user sockets created by ovs use this uid/gid. With the move to TCIB images the default kolla ids were reverted and the previous overwrite dropped. This make e.g. the qemu processes to fail to use the libvirt-vnc bind mounted certificates. This change brings back the previous overwrite of the qemu user uid/gid. * When using specifying ContainerImagePrepare if a tag is explicitly provided in a set, the tag_from_label functionality will not be run as we use the defined tag for the containers. Previously we would still attempt tag lookups even if we wanted a specific tag. Other Notes *********** * *container-images/tripleo_containers.yaml.j2* is now used to determine what containers are used for which services when running the container image prepare process runs. Changes in tripleo-common 11.4.0..11.5.0 ---------------------------------------- ae31ba5e TCIB: Add cinder-volume extend_start.sh script 460b0d57 Move daxio package to be arch specific 10d65543 remove c7 update/upgrdae jobs 2b628273 Fix localization for horizon container efa167dd Enable pylint 0bb0ad88 [TRAIN-Only] Make forks configurable a5791942 Wire up new tripleo upgrades jobs template b657dd95 Fix ImportWarning during importing a module 6e95a99d Add daxio package for nvdimm scenario d6b98f25 [stable] Raise proper error for None network config bcd909bd Changed retry logic on buildah build (train) aa392763 Revert "Add better exception logging for builds" 5ea34d92 Fix wait condition for retry 353b4ef5 Add better exception logging for builds ce4d91e9 Change qemu user id to match previous releases c6365af7 The lower constraint file has been removed 52394870 Move away from "ss" and drop default verbose mode 0630a76c remove lower-constraints 7968e4e4 TCIB: Pick not best when building on centos/ubi 8bae698d Switch to tripleo_dense f7383cb5 Re-add ndctl to nova-compute image e1cbd923 Add heat-engine healthcheck b4f93ead Add retry to build 4f4524e1 Remove centos-7-standalone-upgrade 5a86b8c1 [TRAIN ONLY] Handle empty role Specific Parameter 1de69f16 [train] Switch to content provider jobs/templates 2c79a6ba Add tripleo_states to default callback whitelist 59c5006e Add both validations path for inflight validation 1070bd4c Improve bearer auth handling c5cf036d Adjust Ansible forks caculations eba7b7b0 Add collectd-libpod-stats to image b2045f07 Change image_source from kolla to tripleo 96ec479d Remove the base layers c8493fc8 Switch to tripleo_containers.yaml 07d8f6c9 Lower default verbosity for ansible 23e79805 [Train] Check for existence of stack 9be4f9cb overcloud status report the correct cd for the given plan 87df35c5 Consume ceph, prom, grafana containers from quay.io c214fddb [train-only] Remove duplicated standalone job 75bd92ac Run tripleo-build-containers-ubi-8 on BuildahBuilder code fd017a44 Remove c7 containers multinode job a023e2a7 Support basic auth for image registry 6cd1aabb Include virt-admin in the nova_libvirt container 6ee2e643 Add neutron-mlnx-agent container image e5adc226 Set PROFILE_TASKS_TASK_OUTPUT_LIMIT e53f6f73 Fix handling of default_tag c9dffbd2 update tripleo-train tripleo_containers to default names 6121387e Add interface-names to Rhel8 images d1aa0810 [train-only] add ec2-api image into TCIB 0e83acbb [TRAIN-Only] Add healthcheck debug flag 9b994a2e Remove tripleo.skydive_ansible.v1 workbook 6a650331 Remove skydive support 9fd82c33 [TRAIN-Only] Default centos8 namespace fix 305661bc tcib: rename kolla_version to tcib_managed 844f1d56 First pass at script cleanup 74ab9401 Fix config parser warning a3ccf6fb Relax filter_images_with_labels() and re-add kolla_version 78ed3b00 Add modify_only_with_source bc29d7f9 Wait for lock release during power state change 38c00ad8 [squash] Backport RHEL modules into base image 3f491e5f TCIB: add etcd user to the kolla group 95c03f65 TCIB: add python3-pynacl to mariadb image 4dac8730 container-images: add librados2 to gnocchi-base 365bfd51 tcib: move cradox to gnocchi-base & enable ci on scenario001 5397e9eb (backward compatibility) create kolla UIDs/GIDs for TCIB 662b0acd Ensure atomic inventory file rename runs on the same mountpoint 89827714 Ensure image files are readable to apache 54462799 Remove check for ./puppet/services when filtering services f7a8d7f8 Don't build image upload tasks when dry_run=True 7f1bf79e [TRAIN-Only] Dynamically add centos8 namespace 082d3bed Switch to use virt:8.3 for non-kolla container builds a5159592 Correct the inventory generation 67f9cb1c Ensure tripleo ansible inventory file update is atomic 8b379fb9 Add exponential backoff to ratelimited requests e0c8d777 Revert "Fix stdout_callback" fff6f888 Stop doing tag_from_label lookup with tag 933b52b1 Reduce workers for label lookups 280ed7a5 Properly collect tasks with or conditional in step file. 4dd1bd19 Handle redirects for blobs better 5ea95eca Port jobs from centos7 to centos8 a5b956c5 [train-only] add ed25519 to TCIB (placement image) d2db27cd Add update and post update tasks to step generated file. 6b5fde5e Add a strict option to per role tasks file generation. 2e507941 Switch 55-heat-config to async 344f3b60 Run standalone jobs with locally built images 38a3a5d1 Drop systemd support from nsswitch.conf 2cc35f72 Implement a new Container Images layout b3ec4209 Fix parameter_defaults for nova less deploy 305bb019 Generate a play per step for external_deploy_tasks 19efd014 config-download remove role arg from _write_tasks_per_step() 7cf3af6a [TRAIN-and-older] Reduce wait time to send ansible output 5d651614 Don't disable centos-rabbitmq-38 repo in base image f161bfb0 [Train-Only] Fix mixing nodes and limit_hosts. 258a4af1 Retry fetching {{ deployment_uuid }}.notify.json file 589ac8ac Don't assume default tag exists in container repo d9433a5a ansible: limit_hosts now takes precedence over blacklisted_hostnames 8806829a Catch exception if servers are in error state with no bm_node attached e91a4847 Fix stdout_callback 34283f29 Ensure lanplus is unset when using redfish f36dcdcd [Q->T] Include manila in the Queens to Train Upgrade 6488f98c Avoid tox-in-tox issue 2144ac0b Fix the ANSIBLE_LOG_PATH value. Diffstat (except docs and test files) ------------------------------------- .pre-commit-config.yaml | 8 +- .pylintrc | 66 ++ .../container_image_prepare_defaults.yaml | 14 +- container-images/kolla/barbican-base/sudoers | 1 + container-images/kolla/base/httpd_setup.sh | 20 + container-images/kolla/base/set_configs.py | 436 +++++++++ container-images/kolla/base/start.sh | 18 + container-images/kolla/base/sudoers | 18 + container-images/kolla/base/uid_gid_manage.sh | 126 +++ .../kolla/cinder-volume/cinder-volume-sudoers | 1 + .../kolla/cinder-volume/extend_start.sh | 5 + container-images/kolla/ec2-api/extend_start.sh | 3 + container-images/kolla/glance-api/extend_start.sh | 9 + container-images/kolla/horizon/extend_start.sh | 126 +++ container-images/kolla/iscsid/extend_start.sh | 7 + container-images/kolla/keystone/extend_start.sh | 31 + container-images/kolla/mariadb/extend_start.sh | 35 + .../kolla/mariadb/security_reset.expect | 58 ++ .../kolla/neutron-base/neutron_sudoers | 4 + .../ovn/ovn-nb-db-server/start_nb_db_server.sh | 28 + .../ovn/ovn-sb-db-server/start_sb_db_server.sh | 29 + container-images/kolla/rabbitmq/extend_start.sh | 16 + container-images/kolla/swift-base/swift-rootwrap | 10 + container-images/kolla/swift-base/swift-sudoers | 2 + .../kolla/tripleoclient/create_super_user.sh | 18 + container-images/overcloud_containers.yaml | 14 +- container-images/overcloud_containers.yaml.j2 | 21 +- container-images/tcib/base/base.yaml | 66 ++ container-images/tcib/base/collectd/collectd.yaml | 58 ++ container-images/tcib/base/cron/cron.yaml | 6 + container-images/tcib/base/etcd/etcd.yaml | 9 + container-images/tcib/base/haproxy/haproxy.yaml | 10 + .../tcib/base/keepalived/keepalived.yaml | 6 + container-images/tcib/base/mariadb/mariadb.yaml | 32 + .../tcib/base/memcached/memcached.yaml | 8 + .../tcib/base/multipathd/multipathd.yaml | 6 + .../tcib/base/os/aodh-base/aodh-api/aodh-api.yaml | 11 + .../tcib/base/os/aodh-base/aodh-base.yaml | 6 + .../aodh-base/aodh-evaluator/aodh-evaluator.yaml | 7 + .../os/aodh-base/aodh-listener/aodh-listener.yaml | 7 + .../os/aodh-base/aodh-notifier/aodh-notifier.yaml | 7 + .../barbican-base/barbican-api/barbican-api.yaml | 11 + .../tcib/base/os/barbican-base/barbican-base.yaml | 9 + .../barbican-keystone-listener.yaml | 7 + .../barbican-worker/barbican-worker.yaml | 7 + .../base/os/ceilometer-base/ceilometer-base.yaml | 9 + .../ceilometer-central/ceilometer-central.yaml | 7 + .../ceilometer-compute/ceilometer-compute.yaml | 6 + .../ceilometer-ipmi/ceilometer-ipmi.yaml | 7 + .../ceilometer-notification.yaml | 8 + .../base/os/cinder-base/cinder-api/cinder-api.yaml | 11 + .../cinder-base/cinder-backup/cinder-backup.yaml | 13 + .../tcib/base/os/cinder-base/cinder-base.yaml | 11 + .../cinder-scheduler/cinder-scheduler.yaml | 3 + .../cinder-base/cinder-volume/cinder-volume.yaml | 21 + .../designate-api/designate-api.yaml | 6 + .../designate-backend-bind9.yaml | 6 + .../base/os/designate-base/designate-base.yaml | 9 + .../designate-central/designate-central.yaml | 6 + .../designate-mdns/designate-mdns.yaml | 6 + .../designate-producer/designate-producer.yaml | 6 + .../designate-sink/designate-sink.yaml | 6 + .../designate-worker/designate-worker.yaml | 7 + container-images/tcib/base/os/ec2-api/ec2-api.yaml | 13 + .../tcib/base/os/glance-api/glance-api.yaml | 18 + .../os/gnocchi-base/gnocchi-api/gnocchi-api.yaml | 7 + .../tcib/base/os/gnocchi-base/gnocchi-base.yaml | 14 + .../gnocchi-metricd/gnocchi-metricd.yaml | 7 + .../gnocchi-statsd/gnocchi-statsd.yaml | 7 + .../tcib/base/os/heat-base/heat-all/heat-all.yaml | 8 + .../os/heat-base/heat-api-cfn/heat-api-cfn.yaml | 7 + .../tcib/base/os/heat-base/heat-api/heat-api.yaml | 7 + .../tcib/base/os/heat-base/heat-base.yaml | 10 + .../base/os/heat-base/heat-engine/heat-engine.yaml | 7 + container-images/tcib/base/os/horizon/horizon.yaml | 19 + .../base/os/ironic-base/ironic-api/ironic-api.yaml | 11 + .../tcib/base/os/ironic-base/ironic-base.yaml | 6 + .../ironic-conductor/ironic-conductor.yaml | 30 + .../ironic-inspector/ironic-inspector.yaml | 9 + .../base/os/ironic-base/ironic-pxe/ironic-pxe.yaml | 14 + container-images/tcib/base/os/iscsid/iscsid.yaml | 12 + .../tcib/base/os/keystone/keystone.yaml | 23 + .../base/os/manila-base/manila-api/manila-api.yaml | 10 + .../tcib/base/os/manila-base/manila-base.yaml | 6 + .../manila-scheduler/manila-scheduler.yaml | 3 + .../os/manila-base/manila-share/manila-share.yaml | 14 + .../os/mistral-base/mistral-api/mistral-api.yaml | 11 + .../tcib/base/os/mistral-base/mistral-base.yaml | 9 + .../mistral-engine/mistral-engine.yaml | 7 + .../mistral-event-engine/mistral-event-engine.yaml | 7 + .../mistral-executor/mistral-executor.yaml | 15 + .../ironic-neutron-agent/ironic-neutron-agent.yaml | 8 + .../neutron-agent-base/neutron-agent-base.yaml | 6 + .../neutron-dhcp-agent/neutron-dhcp-agent.yaml | 3 + .../neutron-l3-agent/neutron-l3-agent.yaml | 3 + .../neutron-metadata-agent-ovn.yaml | 7 + .../tcib/base/os/neutron-base/neutron-base.yaml | 18 + .../neutron-metadata-agent.yaml | 3 + .../neutron-mlnx-agent/neutron-mlnx-agent.yaml | 8 + .../neutron-openvswitch-agent.yaml | 7 + .../neutron-server-ovn/neutron-server-ovn.yaml | 12 + .../neutron-server/neutron-server.yaml | 11 + .../neutron-sriov-agent/neutron-sriov-agent.yaml | 7 + .../tcib/base/os/nova-base/nova-api/nova-api.yaml | 12 + .../tcib/base/os/nova-base/nova-base.yaml | 6 + .../nova-compute-ironic/nova-compute-ironic.yaml | 12 + .../os/nova-base/nova-compute/nova-compute.yaml | 31 + .../nova-base/nova-conductor/nova-conductor.yaml | 7 + .../os/nova-base/nova-libvirt/nova-libvirt.yaml | 24 + .../nova-base/nova-novncproxy/nova-novncproxy.yaml | 8 + .../nova-base/nova-scheduler/nova-scheduler.yaml | 8 + .../tcib/base/os/novajoin-base/novajoin-base.yaml | 6 + .../novajoin-notifier/novajoin-notifier.yaml | 2 + .../novajoin-server/novajoin-server.yaml | 2 + .../os/octavia-base/octavia-api/octavia-api.yaml | 12 + .../tcib/base/os/octavia-base/octavia-base.yaml | 6 + .../octavia-health-manager.yaml | 7 + .../octavia-housekeeping/octavia-housekeeping.yaml | 7 + .../octavia-worker/octavia-worker.yaml | 7 + container-images/tcib/base/os/os.yaml | 20 + .../tcib/base/os/panko-api/panko-api.yaml | 12 + .../tcib/base/os/placement-api/placement-api.yaml | 13 + .../os/swift-base/swift-account/swift-account.yaml | 7 + .../tcib/base/os/swift-base/swift-base.yaml | 12 + .../swift-container/swift-container.yaml | 7 + .../os/swift-base/swift-object/swift-object.yaml | 7 + .../swift-proxy-server/swift-proxy-server.yaml | 12 + container-images/tcib/base/os/tempest/tempest.yaml | 8 + .../tcib/base/os/zaqar-wsgi/zaqar-wsgi.yaml | 10 + container-images/tcib/base/ovn-base/ovn-base.yaml | 9 + .../ovn-base/ovn-controller/ovn-controller.yaml | 6 + .../ovn-nb-db-server/ovn-nb-db-server.yaml | 8 + .../tcib/base/ovn-base/ovn-northd/ovn-northd.yaml | 12 + .../ovn-sb-db-server/ovn-sb-db-server.yaml | 8 + .../tcib/base/qdrouterd/qdrouterd.yaml | 11 + container-images/tcib/base/rabbitmq/rabbitmq.yaml | 18 + container-images/tcib/base/redis/redis.yaml | 16 + container-images/tcib/base/rsyslog/rsyslog.yaml | 9 + .../tcib/base/tripleoclient/tripleoclient.yaml | 16 + container-images/tripleo_containers.yaml | 211 ++++ container-images/tripleo_containers.yaml.j2 | 1007 ++++++++++++++++++++ .../tripleo_kolla_template_overrides.j2 | 4 +- healthcheck/common.sh | 70 +- image-yaml/overcloud-hardened-images-rhel8.yaml | 2 + image-yaml/overcloud-images-ceph-rhel8.yaml | 2 + image-yaml/overcloud-images-rhel8.yaml | 2 + lower-constraints.txt | 112 --- .../check_for_default_tag-09fe34d2ac434890.yaml | 6 + ...e-modify_only_with_source-d9be8cc7236e7c94.yaml | 6 + .../notes/healthcheck-debug-0fbcfebd9042720c.yaml | 6 + .../limit_over_blacklist-3ce81ecf04b09997.yaml | 10 + releasenotes/notes/no_ss-368721c3af17b782.yaml | 12 + .../notes/qemu_user_id-32d8f17099a6f002.yaml | 10 + .../remove-skydive-support-1cea22a7419a3b13.yaml | 5 + ...g-lookup-if-tag-specified-2284c45dc0f87693.yaml | 7 + ...e-tripleo-containers-file-0590a59f56fb3907.yaml | 6 + scripts/container-update.py | 6 +- scripts/containerfile-converter.py | 245 +++++ setup.py | 1 + tox.ini | 11 +- tripleo_common/actions/ansible.py | 71 +- tripleo_common/actions/baremetal_deploy.py | 4 +- tripleo_common/actions/container_images.py | 2 +- tripleo_common/actions/deployment.py | 21 +- tripleo_common/actions/parameters.py | 5 +- tripleo_common/actions/undercloud.py | 2 +- tripleo_common/constants.py | 19 +- tripleo_common/exception.py | 1 + tripleo_common/image/builder/buildah.py | 33 +- tripleo_common/image/exception.py | 4 + tripleo_common/image/image_export.py | 4 +- tripleo_common/image/image_uploader.py | 304 ++++-- tripleo_common/image/kolla_builder.py | 74 +- tripleo_common/inventories.py | 13 +- tripleo_common/inventory.py | 11 +- tripleo_common/templates/deployments.yaml | 30 +- tripleo_common/utils/config.py | 80 +- tripleo_common/utils/locks/base.py | 6 + tripleo_common/utils/locks/processlock.py | 3 + tripleo_common/utils/locks/threadinglock.py | 1 + tripleo_common/utils/nodes.py | 15 +- workbooks/access.yaml | 13 +- workbooks/baremetal.yaml | 4 +- workbooks/deployment.yaml | 9 +- workbooks/package_update.yaml | 4 +- workbooks/plan_management.yaml | 2 +- workbooks/skydive-ansible.yaml | 74 -- zuul.d/layout.yaml | 119 ++- 205 files changed, 6185 insertions(+), 826 deletions(-)