We are thrilled to announce the release of: trove 14.0.0: OpenStack DBaaS This release is part of the victoria release series. The source is available from: https://opendev.org/openstack/trove Download the package from: https://tarballs.openstack.org/trove/ Please report issues through: https://bugs.launchpad.net/trove/+bugs For more details, please see below. Changes in trove 13.0.0..14.0.0 ------------------------------- 6ccdb4b8 Handle case where no networks are selected at create 21011948 Only confirm resize when nova instance status is VERIFY_RESIZE aa3c53e6 [Postgresql] Adding config parameters ac664f23 Update TOX_CONSTRAINTS_FILE for stable/victoria 593ac3a8 Update .gitreview for stable/victoria 28453260 Set limit to -1 when getting Nova instances 6ec38419 Set default limit for novaclient 7aa26958 Install pip3 for image build aa023d14 [Doc] More description for nics for creating instance d75151bb [Doc] Improve building Trove guest image instruction 7537b325 [Backup] Ignore 404 error when deleting swift objects 3aa68472 Change access column type for instance 86a0102a Add project_id to backup response f79bb53a Support getting backups of a specific project 7d2e6b7c [Doc] Refine the root enable API description 72e20e4e Only enable user and database API for MySQL df4f7476 Fix some API calls for PostgreSQL 5482c546 [Postgresql] Create replica 78d1a376 [API doc] Add replica_of param for getting instance 4fb41b51 Postgresql: Backup and restore ec8a10e3 Use sudo when deleting image cache dir 5bf159de Remove six.PY3 00a07f24 Use pip3 instead of pip in trovestack 43d7aec7 Improve docker image for database backup/restore ca4e029f Remove the definition of trove-grenade 77ca2079 [API doc] Add more description for deleting backups d0cfb41b PostgreSQL support 5e1cd054 [API doc] Improve description of swift_container for creating backups 8daade00 Support to check if subnet is associated with router 9e7c4c08 Trove upgrade guide 7486a7c3 Fix access field for instances created before upgrade 9914bd6a Allow to specify root password when building guest image c4fd9cbe Fix functional test for instance access operation 4de40cb5 Support to update instance access 680a4300 Show instance access information 722ac4bc Wait for instance after rebuild 6b1c31e2 Add instruction for upgrading trove guest agent cf3e9a6e Support to rebuild instance by admin 8e48d757 Resize replicas (volume) together with primary efb6a811 Using same config with primary for replicas 30955521 Fix oslo-config-generator command example in doc ba046b2a Support online volume resize 56bb6f39 Monkey patch original current_thread _active 058490a6 Fix the slow start of mariadb container 828e8738 Support backup strategy API f8ca333b Correct some information in docs d8154248 Doc: Fix configuration API d2643794 Update location of file validation-rules.json 428e3644 Support image type for guest image building 4b654a84 Use unittest.mock instead of mock 4bf6c496 Release note for adding replicas in list instance response fb70d24c Show replicas info for listing instances 53541724 Support subnet and IP for instance creation b77f7b9f Switch to newer openstackdocstheme and reno versions dc117d8d Set status to ERROR if heartbeat expires 816a7bca Stop to use the __future__ module. a7af0e4d Support to remove datastore config parameters 87b65e3c Support to generate config sample file 5011892b Doc: update trove design fb70c2e2 Add doc for how to run functional test 761fa7f8 Fix job publish-trove-guest-image-ubuntu-bionic-dev e2b26fdd Trove doc and CI 8cd3670b Remove translation sections from setup.cfg 9c5f5c90 Fix hacking min version to 3.0.1 38220570 Update docs upgrade datastore aa1d4d22 Datastore containerization ed726fd2 Cap jsonschema 3.2.0 as the minimal version 523d66e8 Fix CI job ae4d1a5e Add command example for preparing datastore 8887503e Add py38 package metadata d4449826 Add troubleshooting guide 91051912 Remove flavor API feb98254 Remove volume-types API 74895a5c Use unittest.mock instead of third party mock 429c3989 Support ip address type for instances f32c4c8f Add Python3 victoria unit tests db76fe96 Update master for stable/ussuri Diffstat (except docs and test files) ------------------------------------- .gitreview | 1 + .zuul.yaml | 97 +- api-ref/source/backup-strategy.inc | 111 + api-ref/source/backups.inc | 13 +- api-ref/source/conf.py | 9 +- api-ref/source/databases.inc | 5 +- api-ref/source/index.rst | 1 + api-ref/source/instance-actions.inc | 33 + api-ref/source/instances.inc | 123 +- api-ref/source/parameters.yaml | 75 +- .../samples/backup-strategy-create-request.json | 6 + .../samples/backup-strategy-create-response.json | 8 + .../samples/backup-strategy-list-response.json | 10 + .../samples/config-group-create-request.json | 9 +- .../samples/config-group-create-response.json | 28 +- .../source/samples/instance-create-response.json | 12 +- .../samples/instance-list-detail-response.json | 34 +- api-ref/source/samples/instance-list-response.json | 12 +- ...tance-mgmt-action-rebuild-instance-request.json | 5 + .../samples/instance-mgmt-list-response.json | 34 +- .../samples/instance-mgmt-show-response.json | 22 +- .../instance-patch-detach-replica-request.json | 3 +- api-ref/source/samples/instance-show-response.json | 22 +- .../samples/instance-update-access-request.json | 8 + .../source/samples/user-root-enable-request.json | 3 + api-ref/source/users.inc | 20 +- babel.cfg | 2 - backup/Dockerfile | 29 + .../storage/experimental => backup}/__init__.py | 0 .../experimental => backup/drivers}/__init__.py | 0 backup/drivers/base.py | 207 ++ backup/drivers/innobackupex.py | 136 + backup/drivers/mariabackup.py | 86 + backup/drivers/mysql_base.py | 143 + backup/drivers/postgres.py | 249 ++ backup/install.sh | 33 + backup/main.py | 166 + backup/requirements.txt | 8 + .../cassandra => backup/storage}/__init__.py | 0 backup/storage/base.py | 48 + backup/storage/swift.py | 298 ++ backup/utils/__init__.py | 46 + backup/utils/postgresql.py | 53 + devstack/plugin.sh | 25 +- devstack/settings | 16 +- etc/trove/cloudinit/README | 3 - etc/trove/conf.d/README | 4 - etc/trove/conf.d/guest_info.conf | 1 - etc/trove/trove-guestagent.conf.sample | 166 - etc/trove/trove-workbook.yaml | 19 - etc/trove/trove.conf.sample | 318 -- etc/trove/trove.conf.test | 1 - integration/scripts/conf.json.example | 12 - integration/scripts/conf/test_begin.conf | 2 +- .../scripts/files/elements/apt-conf-dir/README.rst | 16 - .../extra-data.d/99-use-host-apt-confd | 21 - .../files/elements/guest-agent/element-deps | 1 + .../99-reliable-apt-key-importing.bash | 34 - .../31-guest-agent-install | 51 + .../75-guest-agent-install | 45 - .../guest-agent-dev.service | 31 + .../guest-agent-source-install/guest-agent.service | 7 +- .../elements/guest-agent/package-installs.yaml | 54 +- ...agent-systemd => 31-enable-guest-agent-systemd} | 0 .../files/elements/no-resolvconf/README.rst | 8 - .../finalise.d/99-disable-resolv-conf | 19 - .../scripts/files/elements/root-passwd/README.rst | 5 + .../elements/root-passwd/post-install.d/99-setup | 11 + .../files/elements/ubuntu-docker/element-deps | 1 + .../elements/ubuntu-docker/install.d/21-docker | 19 + .../99-reliable-apt-key-importing.bash | 34 - .../ubuntu-guest/extra-data.d/11-ssh-key-dev | 17 + .../ubuntu-guest/extra-data.d/15-trove-dep | 28 - .../elements/ubuntu-guest/extra-data.d/62-ssh-key | 27 - .../elements/ubuntu-guest/install.d/05-base-apps | 10 - .../50-user => ubuntu-guest/install.d/11-user} | 2 +- .../elements/ubuntu-guest/install.d/12-ssh-key-dev | 22 + .../elements/ubuntu-guest/install.d/15-trove-dep | 30 - .../files/elements/ubuntu-guest/install.d/50-user | 18 - .../elements/ubuntu-guest/install.d/62-ssh-key | 25 - .../files/elements/ubuntu-guest/install.d/98-ssh | 8 - .../elements/ubuntu-guest/install.d/99-clean-apt | 11 - .../{05-ipforwarding => 11-ipforwarding} | 0 .../ubuntu-guest/post-install.d/{10-ntp => 12-ntp} | 0 ...-trove-guest-sudoers => 13-trove-guest-sudoers} | 0 .../ubuntu-guest/post-install.d/90-apt-get-update | 9 - .../ubuntu-guest/pre-install.d/04-baseline-tools | 7 - .../ubuntu-guest/pre-install.d/11-baseline-tools | 7 + .../files/elements/ubuntu-mariadb/README.md | 3 - .../pre-install.d/20-apparmor-mysql-local | 11 - .../ubuntu-mysql/pre-install.d/10-percona-apt-key | 6 +- .../pre-install.d/20-apparmor-mysql-local | 12 - .../ubuntu-postgresql/install.d/30-postgresql | 25 - .../pre-install.d/10-postgresql-repo | 14 - .../elements/ubuntu-xenial-guest/element-deps | 1 - .../extra-data.d/20-guest-systemd | 21 - .../elements/ubuntu-xenial-guest/install.d/20-etc | 8 - .../install.d/21-use-ubuntu-certificates | 12 - .../install.d/22-decrease-networking-timeout | 7 + .../post-install.d/91-hwe-kernel | 29 - .../ubuntu-xenial-guest/pre-install.d/01-trim-pkgs | 90 - .../elements/ubuntu-xenial-mariadb/element-deps | 1 - .../ubuntu-xenial-mariadb/install.d/30-mariadb | 39 - .../elements/ubuntu-xenial-mysql/element-deps | 1 - .../ubuntu-xenial-mysql/install.d/30-mysql | 39 - .../elements/ubuntu-xenial-postgresql/element-deps | 1 - integration/scripts/functions_qemu | 96 +- .../scripts/local.conf.d/ceilometer_cinder.conf.rc | 3 - .../scripts/local.conf.d/ceilometer_nova.conf.rc | 3 - .../local.conf.d/ceilometer_services.conf.rc | 3 - integration/scripts/local.conf.d/sample.rc | 42 - .../scripts/local.conf.d/trove_services.conf.rc | 24 - integration/scripts/local.conf.d/use_kvm.rc | 4 - integration/scripts/local.conf.d/use_uuid_token.rc | 3 - integration/scripts/local.conf.d/using_vagrant.rc | 9 - integration/scripts/local.conf.rc | 37 - integration/scripts/trovestack | 164 +- lower-constraints.txt | 11 +- playbooks/image-build/run.yaml | 3 +- playbooks/legacy/grenade-dsvm-trove/post.yaml | 15 - playbooks/legacy/grenade-dsvm-trove/run.yaml | 47 - .../ussuri-add-ip-addresses-for-instance.yaml | 8 + .../victoria-check-subnet-router-association.yaml | 7 + .../notes/victoria-database-containerization.yaml | 15 + .../notes/victoria-expired-database-status.yaml | 4 + .../notes/victoria-list-project-backups.yaml | 3 + releasenotes/notes/victoria-rebuild-instance.yaml | 8 + .../victoria-resize-vollume-for-replication.yaml | 4 + .../victoria-show-replicas-for-instance-list.yaml | 4 + .../victoria-show-update-instance-access.yaml | 3 + .../notes/victoria-support-backup-strategy.yaml | 6 + .../notes/victoria-support-online-resize.yaml | 7 + .../victoria-support-subnet-and-ip-address.yaml | 7 + ...ictoria-trove-manager-remove-config-params.yaml | 5 + releasenotes/source/conf.py | 10 +- releasenotes/source/index.rst | 1 + releasenotes/source/ussuri.rst | 6 + requirements.txt | 5 +- roles/trove-devstack/defaults/main.yml | 2 +- setup.cfg | 16 +- test-requirements.txt | 8 +- tools/install_venv.py | 2 - tools/trove-pylint.config | 18 + tools/trove-pylint.py | 2 - tox.ini | 10 +- trove/backup/models.py | 79 +- trove/backup/service.py | 77 +- trove/backup/views.py | 29 + trove/cluster/views.py | 6 +- trove/cmd/__init__.py | 6 + trove/cmd/guest.py | 32 +- trove/cmd/manage.py | 26 +- trove/common/api.py | 43 +- trove/common/apischema.py | 99 +- trove/common/base_wsgi.py | 2 - trove/common/cfg.py | 322 +- trove/common/clients_admin.py | 48 +- trove/common/exception.py | 41 +- trove/common/neutron.py | 103 +- trove/common/notification.py | 17 - trove/common/policies/backups.py | 44 +- trove/common/policies/base.py | 2 + trove/common/schemas/atom-link.rng | 141 - trove/common/schemas/atom.rng | 597 ---- trove/common/schemas/v1.1/limits.rng | 28 - .../cluster/experimental/mongodb/taskmanager.py | 9 +- trove/common/strategies/storage/base.py | 44 - trove/common/strategies/storage/swift.py | 302 -- trove/common/stream_codecs.py | 15 +- trove/common/utils.py | 10 +- trove/conductor/manager.py | 6 +- trove/configuration/models.py | 11 +- trove/configuration/service.py | 6 +- trove/db/sqlalchemy/mappers.py | 2 + trove/db/sqlalchemy/migrate_repo/schema.py | 1 - .../versions/045_add_backup_strategy.py | 46 + .../versions/046_add_access_to_instance.py} | 17 +- trove/dns/designate/driver.py | 4 +- trove/extensions/common/models.py | 27 +- trove/extensions/common/service.py | 10 +- trove/extensions/mgmt/instances/models.py | 3 +- trove/extensions/mgmt/instances/service.py | 11 +- trove/flavor/service.py | 54 - trove/guestagent/api.py | 60 +- trove/guestagent/backup/__init__.py | 45 - trove/guestagent/backup/backupagent.py | 178 - trove/guestagent/common/configuration.py | 27 +- trove/guestagent/common/guestagent_utils.py | 41 + trove/guestagent/common/operating_system.py | 141 +- .../datastore/experimental/cassandra/manager.py | 368 -- .../datastore/experimental/cassandra/service.py | 1314 ------- .../datastore/experimental/couchbase/manager.py | 122 - .../datastore/experimental/couchbase/service.py | 268 -- .../datastore/experimental/couchbase/system.py | 50 - .../datastore/experimental/couchdb/manager.py | 167 - .../datastore/experimental/couchdb/service.py | 584 --- .../datastore/experimental/couchdb/system.py | 74 - .../datastore/experimental/db2/manager.py | 157 - .../datastore/experimental/db2/service.py | 626 ---- .../datastore/experimental/db2/system.py | 89 - .../datastore/experimental/mariadb/__init__.py | 0 .../datastore/experimental/mariadb/manager.py | 29 - .../datastore/experimental/mariadb/service.py | 109 - .../datastore/experimental/mongodb/__init__.py | 0 .../datastore/experimental/mongodb/manager.py | 268 -- .../datastore/experimental/mongodb/service.py | 843 ----- .../datastore/experimental/mongodb/system.py | 46 - .../datastore/experimental/percona/__init__.py | 0 .../datastore/experimental/percona/manager.py | 36 - .../datastore/experimental/percona/service.py | 84 - .../datastore/experimental/postgresql/__init__.py | 0 .../datastore/experimental/postgresql/manager.py | 344 -- .../datastore/experimental/postgresql/service.py | 1058 ------ .../datastore/experimental/pxc/__init__.py | 0 .../datastore/experimental/pxc/manager.py | 27 - .../datastore/experimental/pxc/service.py | 56 - .../datastore/experimental/redis/__init__.py | 0 .../datastore/experimental/redis/manager.py | 345 -- .../datastore/experimental/redis/service.py | 561 --- .../datastore/experimental/redis/system.py | 37 - .../datastore/experimental/vertica/__init__.py | 0 .../datastore/experimental/vertica/manager.py | 161 - .../datastore/experimental/vertica/service.py | 618 ---- .../datastore/experimental/vertica/system.py | 125 - .../guestagent/datastore/galera_common/__init__.py | 0 .../guestagent/datastore/galera_common/manager.py | 80 - .../guestagent/datastore/galera_common/service.py | 93 - trove/guestagent/datastore/manager.py | 574 ++- .../couchbase => mariadb}/__init__.py | 0 trove/guestagent/datastore/mariadb/manager.py | 26 + trove/guestagent/datastore/mariadb/service.py | 88 + trove/guestagent/datastore/mysql/manager.py | 38 +- trove/guestagent/datastore/mysql/service.py | 90 +- trove/guestagent/datastore/mysql_common/manager.py | 551 ++- trove/guestagent/datastore/mysql_common/service.py | 852 ++--- .../{experimental/couchdb => postgres}/__init__.py | 0 trove/guestagent/datastore/postgres/manager.py | 188 + .../pgsql_query.py => postgres/query.py} | 45 +- trove/guestagent/datastore/postgres/service.py | 760 ++++ trove/guestagent/datastore/service.py | 297 +- .../datastore/technical-preview/__init__.py | 0 trove/guestagent/dbaas.py | 29 +- trove/guestagent/guest_log.py | 39 +- trove/guestagent/module/module_manager.py | 2 +- trove/guestagent/strategies/backup/__init__.py | 25 - trove/guestagent/strategies/backup/base.py | 145 - .../strategies/backup/experimental/__init__.py | 0 .../backup/experimental/cassandra_impl.py | 117 - .../backup/experimental/couchbase_impl.py | 108 - .../strategies/backup/experimental/couchdb_impl.py | 35 - .../strategies/backup/experimental/db2_impl.py | 176 - .../strategies/backup/experimental/mariadb_impl.py | 112 - .../strategies/backup/experimental/mongo_impl.py | 106 - .../backup/experimental/postgresql_impl.py | 257 -- .../strategies/backup/experimental/redis_impl.py | 39 - trove/guestagent/strategies/backup/mysql_impl.py | 150 - .../guestagent/strategies/replication/__init__.py | 2 +- trove/guestagent/strategies/replication/base.py | 4 +- .../replication/experimental/__init__.py | 0 .../replication/experimental/postgresql_impl.py | 306 -- .../replication/experimental/redis_sync.py | 91 - .../replication/{experimental => }/mariadb_gtid.py | 58 +- .../strategies/replication/mysql_base.py | 107 +- .../strategies/replication/mysql_binlog.py | 79 - .../strategies/replication/mysql_gtid.py | 63 +- .../strategies/replication/postgresql.py | 220 ++ trove/guestagent/strategies/restore/__init__.py | 24 - trove/guestagent/strategies/restore/base.py | 113 - .../strategies/restore/experimental/__init__.py | 0 .../restore/experimental/cassandra_impl.py | 69 - .../restore/experimental/couchbase_impl.py | 199 -- .../restore/experimental/couchdb_impl.py | 41 - .../strategies/restore/experimental/db2_impl.py | 90 - .../restore/experimental/mariadb_impl.py | 160 - .../strategies/restore/experimental/mongo_impl.py | 51 - .../restore/experimental/postgresql_impl.py | 202 -- .../strategies/restore/experimental/redis_impl.py | 74 - trove/guestagent/strategies/restore/mysql_impl.py | 364 -- .../experimental/db2 => utils}/__init__.py | 0 trove/guestagent/utils/docker.py | 153 + trove/guestagent/utils/mysql.py | 85 + trove/guestagent/volume.py | 29 +- trove/instance/models.py | 483 ++- trove/instance/service.py | 250 +- .../instance.py => instance/service_status.py} | 2 + trove/instance/tasks.py | 4 + trove/instance/views.py | 54 +- trove/limits/views.py | 4 +- trove/quota/quota.py | 4 +- trove/taskmanager/api.py | 20 +- trove/taskmanager/manager.py | 147 +- trove/taskmanager/models.py | 766 ++-- trove/templates/mariadb/config.template | 1 - trove/templates/mariadb/replica.config.template | 3 +- .../mariadb/replica_source.config.template | 2 + trove/templates/mysql/config.template | 1 - trove/templates/mysql/validation-rules.json | 1250 +++---- trove/templates/postgresql/config.template | 369 +- trove/templates/postgresql/validation-rules.json | 1896 +++++----- .../scenario/runners/configuration_runners.py | 8 +- .../cluster/test_cluster_pxc_controller.py | 6 +- .../cluster/test_cluster_redis_controller.py | 6 +- .../cluster/test_cluster_vertica_controller.py | 6 +- .../configuration/test_configuration_controller.py | 2 +- .../datastore/test_datastore_version_metadata.py | 2 +- .../domain-name-service/test_designate_driver.py | 11 +- .../guestagent/test_agent_heartbeats_models.py | 235 -- .../guestagent/test_galera_cluster_api.py | 151 - .../volume_type/test_volume_type_views.py | 2 +- trove/volume_type/service.py | 36 - 440 files changed, 10610 insertions(+), 36576 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index c53d8f7c..6579d13f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -27 +27 @@ iso8601>=0.1.11 # MIT -jsonschema>=2.6.0 # MIT +jsonschema>=3.2.0 # MIT @@ -40 +39,0 @@ PyMySQL>=0.7.6 # MIT License -Babel!=2.4.0,>=2.3.4 # BSD @@ -50,0 +50,2 @@ diskimage-builder!=1.6.0,!=1.7.0,!=1.7.1,>=1.1.2 # Apache-2.0 +docker>=4.2.0 # Apache-2.0 +psycopg2-binary>=2.6.2 # LGPL/ZPL diff --git a/test-requirements.txt b/test-requirements.txt index c8dbd865..3b90a3fa 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -5 +5 @@ -hacking>=3.0,<3.1.0 # Apache-2.0 +hacking>=3.0.1,<3.1.0 # Apache-2.0 @@ -15 +14,0 @@ python-troveclient>=2.2.0 # Apache-2.0 -mock>=2.0.0 # BSD @@ -19 +17,0 @@ redis>=2.10.0 # MIT -psycopg2>=2.6.2 # LGPL/ZPL @@ -29 +27 @@ tenacity>=4.9.0 # Apache-2.0 -openstackdocstheme>=1.32.1 # Apache-2.0 +openstackdocstheme>=2.2.1 # Apache-2.0 @@ -31 +29 @@ os-api-ref>=1.4.0 # Apache-2.0 -reno>=2.5.0 # Apache-2.0 +reno>=3.1.0 # Apache-2.0