We contentedly announce the release of: karbor 0.6.0: Application Data Protection as a Service for OpenStack This release is part of the queens release series. The source is available from: https://git.openstack.org/cgit/openstack/karbor Download the package from: https://tarballs.openstack.org/karbor/ Please report issues through launchpad: https://bugs.launchpad.net/karbor For more details, please see below. Changes in karbor 0.5.0..0.6.0 ------------------------------ 84f83a3 jsonschema validation: base schema framework and plans API 05be900 Fix server restore failed if network rebuild 72f9965 Replace DbMigrationError with DBMigrationError 9928359 Fix 'limit' and 'marker' error in provider listing 08a9e60 Fix the errors of 'ubuntu-install.rst' 6b565a1 Remove duplicated unit tests of restore API 41afaad Add spec about the checkpoint copy API cf1fb36 Add the checkpoint copy API for Karbor 8ef0bf9 adding more protection parameters retention period 2052c63 Updated from global requirements 10de69e Add try catch exception for plan destory before quota reserve 9484795 Add API controller for the quotas of Karbor 93757df Imported Translations from Zanata 9d7af44 Add tables and data modules for the quotas of Karbor c59f1af Add time format check in API b72463d Zuul: add file extension to playbook path b9055ec Remove policy.json from unit test 2bd39f6 Add verify action for the image backup protection plugin fff8e49 Add verify action for the network backup protection plugin 576cbb3 Add verify action for the pod backup protection plugin d67eb91 Add verify action for the share snapshot protection plugin 7217c66 Add verify action for the database backup protection plugin a78cfa0 Add verify action for the server backup protection plugin 8fa5d82 Add service management API to Karbor 9d359ae Spec: Adding more protection parameters retention period ae39095 Add verify action for the volume snapshot protection plugin 22ed8a9 Add verification API for Karbor fde94b2 Add tenant isolation of checkpoints a9e872b Add backup volume by glance protection plugin 3375686 Add data module for checkpoint verification API c8af5d2 Operation Engine: Support Trigger Multi Node fce73f5 Spec: Add service management API to Karbor ffb0ec5 Add database table for checkpoint verification API 607e787 Updated from global requirements fbcd140 Move legacy jobs to project 0db9e5b Add sample policy file to documentation 1be7ca6 Spec: Add tenant isolation of checkpoints 105f68f Fix the unit tests about policy and the DocumentedRule description 5f69bb8 Add default policy in code for the operation log resource 18b5951 Update unit tests of freezer protection plugin ee52708 Protectable: warn about missing protectables 0bcfd36 devstack: add trustee conf regardless of keystone 3afb417 Fix getting image status error 2f529a6 Short-circuit notifications when not enabled 2c08b82 Add default policy in code for the scheduled operation resource 6afc133 Add default policy in code for the trigger resource 7900222 Add default policy in code for the provider resource 0c5ef4a Add default policy in code for the protectable resource c31902b Add default policy in code for the restore resource 3b1c7a2 Fix the warning about test command 'rm' found but not installed 83f97e4 Add default policy in code for the plan resource 9a4c740 Indicating the location tests directory in oslo_debug_helper 3a9ef00 Correct karbor source install doc 2e0fda1 Updated from global requirements 53958f2 Spec: Add backup volume by glance protection plugin 341321b Updated from global requirements 34e99f8 Add the pod with cinder volumes protection plugin 892f96c Add pod to the parent resource types of volume protectable plugins 9675e96 devstack: run configure with any karbor service 2f037cc Spec: Add checkpoint verification API for Karbor 1f364ae Fix NoneType error when got non-exist instance of protectable. 2dbc1af Updated from global requirements 73058d0 Fix implicit reason for 'provider_id'. ae495b7 Fix improper type for some like UUID fileds in objects. bf752c4 Add freezer protection plugin for karbor dfd09a3 Spec: Support kubernetes pod with persistent volumes protection plugin e9160d2 Updated from global requirements 889915f Use FakeCheckpoint instead of CheckpointCollection 45f20e0 Add kubernetes pod protectable plugin to Karbor 911ceab Fix checkpoints pagination error 1364658 [Trivialfix]Fix typos in karbor 4b53170 Add kubernetes client to Karbor 5f56e8f Add project_id in neutron operations 7a7d7a1 Fix restore security group error 0c04206 Remove network dependency of server resource 334fc98 Fix to use "." to source script files bda0375 Updated from global requirements 0677ee1 Optimize the execution time of karbor fullstack tests e576d34 Add freezer protection plugin for karbor 7d0cd8e Enable nova n-api-meta service in devstack of Karbor fad0e7e Update reno for stable/pike ed8e779 Use PortOpt for port options cdd4be7 Add 'rm -f .testrepository/times.dbm' command in testenv 0acd851 Spec: Implement policies in code Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 19 + devstack/devstackgaterc | 2 +- devstack/local.conf.sample | 2 +- devstack/plugin.sh | 59 +- devstack/settings | 3 +- .../contributor/protection_plugins_guide.rst | 2 +- ...more-protection-parameters-retention-period.rst | 170 +++++ .../specs/kubernetes-pods-protection-plugin.rst | 216 ++++++ .../specs/volume-glance-protection-plugin.rst | 186 +++++ etc/README-policy.yaml.txt | 4 + etc/karbor-policy-generator.conf | 3 + etc/policy.json | 42 -- etc/providers.d/openstack-kubernetes.conf | 21 + karbor/api/schemas/__init__.py | 0 karbor/api/schemas/plans.py | 59 ++ karbor/api/v1/copies.py | 126 ++++ karbor/api/v1/operation_logs.py | 25 +- karbor/api/v1/plans.py | 76 +- karbor/api/v1/protectables.py | 24 +- karbor/api/v1/providers.py | 32 +- karbor/api/v1/quota_classes.py | 125 ++++ karbor/api/v1/quotas.py | 194 +++++ karbor/api/v1/restores.py | 28 +- karbor/api/v1/router.py | 42 ++ karbor/api/v1/scheduled_operations.py | 15 +- karbor/api/v1/services.py | 128 ++++ karbor/api/v1/triggers.py | 31 +- karbor/api/v1/verifications.py | 286 ++++++++ karbor/api/validation/__init__.py | 43 ++ karbor/api/validation/parameter_types.py | 165 +++++ karbor/api/validation/validators.py | 214 ++++++ karbor/cmd/manage.py | 2 +- karbor/common/config.py | 8 + karbor/common/constants.py | 23 +- karbor/common/opts.py | 1 + karbor/context.py | 40 +- karbor/db/api.py | 212 ++++++ karbor/db/sqlalchemy/api.py | 785 +++++++++++++++++++- .../migrate_repo/versions/001_karbor_init.py | 14 + .../versions/002_add_verification_table.py | 39 + .../migrate_repo/versions/003_add_quotas_table.py | 84 +++ karbor/db/sqlalchemy/models.py | 103 ++- karbor/exception.py | 77 ++ karbor/objects/__init__.py | 1 + karbor/objects/checkpoint_record.py | 2 +- karbor/objects/scheduled_operation.py | 4 +- karbor/objects/service.py | 11 + karbor/objects/trigger.py | 2 +- karbor/objects/verification.py | 134 ++++ karbor/policies/__init__.py | 47 ++ karbor/policies/base.py | 34 + karbor/policies/copies.py | 39 + karbor/policies/operation_logs.py | 49 ++ karbor/policies/plans.py | 82 +++ karbor/policies/protectables.py | 72 ++ karbor/policies/providers.py | 94 +++ karbor/policies/quota_classes.py | 49 ++ karbor/policies/quotas.py | 71 ++ karbor/policies/restores.py | 71 ++ karbor/policies/scheduled_operations.py | 71 ++ karbor/policies/services.py | 47 ++ karbor/policies/triggers.py | 82 +++ karbor/policies/verifications.py | 60 ++ karbor/policy.py | 144 +++- karbor/quota.py | 814 +++++++++++++++++++++ karbor/rpc.py | 34 +- .../engine/triggers/timetrigger/time_trigger.py | 293 ++++---- .../timetrigger/timeformats/calendar_time.py | 35 +- .../timetrigger/timeformats/crontab_time.py | 13 +- karbor/services/operationengine/manager.py | 4 + .../operations/protect_operation.py | 2 +- .../operations/retention_operation.py | 192 +++++ karbor/services/operationengine/rpcapi.py | 9 +- karbor/services/protection/api.py | 12 +- karbor/services/protection/bank_plugin.py | 36 +- karbor/services/protection/checkpoint.py | 95 ++- karbor/services/protection/clients/freezer.py | 74 ++ karbor/services/protection/clients/k8s.py | 54 ++ karbor/services/protection/flows/copy.py | 191 +++++ karbor/services/protection/flows/utils.py | 27 +- karbor/services/protection/flows/verify.py | 92 +++ karbor/services/protection/flows/worker.py | 25 + karbor/services/protection/manager.py | 149 +++- karbor/services/protection/protectable_plugin.py | 7 +- .../protection/protectable_plugins/network.py | 59 +- .../services/protection/protectable_plugins/pod.py | 115 +++ .../protection/protectable_plugins/volume.py | 70 +- karbor/services/protection/protectable_registry.py | 20 +- karbor/services/protection/protection_plugin.py | 22 + .../database/database_backup_plugin.py | 49 ++ .../database/database_backup_plugin_schemas.py | 6 + .../image/image_plugin_schemas.py | 6 + .../image/image_protection_plugin.py | 139 ++-- .../network/network_plugin_schemas.py | 6 + .../network/neutron_protection_plugin.py | 55 +- .../protection/protection_plugins/pod/__init__.py | 0 .../protection_plugins/pod/pod_plugin_schemas.py | 44 ++ .../pod/pod_protection_plugin.py | 366 +++++++++ .../server/nova_protection_plugin.py | 46 +- .../server/server_plugin_schemas.py | 11 + .../share/share_snapshot_plugin.py | 49 ++ .../share/share_snapshot_plugin_schemas.py | 6 + .../protection/protection_plugins/utils.py | 74 ++ .../volume/cinder_protection_plugin.py | 47 ++ .../volume/volume_freezer_plugin.py | 471 ++++++++++++ .../volume/volume_freezer_plugin_schemas.py | 51 ++ .../volume/volume_glance_plugin.py | 487 ++++++++++++ .../volume/volume_glance_plugin_schemas.py | 51 ++ .../volume/volume_plugin_cinder_schemas.py | 6 + .../volume/volume_snapshot_plugin.py | 49 ++ .../volume/volume_snapshot_plugin_schemas.py | 6 + karbor/services/protection/provider.py | 32 +- karbor/services/protection/resource_flow.py | 10 + karbor/services/protection/rpcapi.py | 20 +- .../triggers/timetrigger/test_time_trigger.py | 156 ++-- .../operations/test_retention_operation.py | 197 +++++ .../plugins/test_network_protectable_plugin.py | 3 +- .../unit/plugins/test_pod_protectable_plugin.py | 88 +++ .../unit/plugins/test_volume_protectable_plugin.py | 101 ++- .../unit/protection/test_checkpoint_collection.py | 43 +- .../test_cinder_freezer_protection_plugin.py | 209 ++++++ .../unit/protection/test_cinder_glance_plugin.py | 229 ++++++ .../protection/test_cinder_protection_plugin.py | 31 + .../test_cinder_snapshot_protection_plugin.py | 41 +- .../protection/test_database_protection_plugin.py | 28 +- .../protection/test_glance_protection_plugin.py | 22 +- .../protection/test_manila_protection_plugin.py | 28 +- .../protection/test_neutron_protection_plugin.py | 20 +- .../unit/protection/test_nova_protection_plugin.py | 17 + .../unit/protection/test_pod_protection_plugin.py | 194 +++++ karbor/utils.py | 84 +++ playbooks/legacy/karbor-dsvm-fullstack/post.yaml | 80 ++ playbooks/legacy/karbor-dsvm-fullstack/run.yaml | 87 +++ releasenotes/source/index.rst | 1 + .../locale/en_GB/LC_MESSAGES/releasenotes.po | 61 ++ releasenotes/source/pike.rst | 6 + requirements.txt | 41 +- setup.cfg | 12 + test-requirements.txt | 14 +- tools/install_venv.py | 2 +- tox.ini | 12 +- 189 files changed, 13284 insertions(+), 920 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 5049753..b73be98 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5 +5 @@ -botocore>=1.0.0 # Apache-2.0 +botocore>=1.5.1 # Apache-2.0 @@ -10 +10 @@ eventlet!=0.18.3,!=0.20.1,<0.21.0,>=0.18.2 # MIT -greenlet>=0.3.2 # MIT +greenlet>=0.4.10 # MIT @@ -12,10 +12,12 @@ icalendar>=3.10 # BSD -keystoneauth1>=3.1.0 # Apache-2.0 -keystonemiddleware>=4.12.0 # Apache-2.0 -python-manilaclient>=1.12.0 # Apache-2.0 -oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 -oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.context>=2.14.0 # Apache-2.0 -oslo.db>=4.24.0 # Apache-2.0 -oslo.log>=3.22.0 # Apache-2.0 -oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 -oslo.middleware>=3.27.0 # Apache-2.0 +jsonschema<3.0.0,>=2.6.0 # MIT +keystoneauth1>=3.2.0 # Apache-2.0 +keystonemiddleware>=4.17.0 # Apache-2.0 +kubernetes>=1.0.0 # Apache-2.0 +python-manilaclient>=1.16.0 # Apache-2.0 +oslo.config>=4.6.0 # Apache-2.0 +oslo.concurrency>=3.20.0 # Apache-2.0 +oslo.context>=2.19.2 # Apache-2.0 +oslo.db>=4.27.0 # Apache-2.0 +oslo.log>=3.30.0 # Apache-2.0 +oslo.messaging>=5.29.0 # Apache-2.0 +oslo.middleware>=3.31.0 # Apache-2.0 @@ -23,4 +25,4 @@ oslo.policy>=1.23.0 # Apache-2.0 -oslo.serialization!=2.19.1,>=1.10.0 # Apache-2.0 -oslo.service>=1.10.0 # Apache-2.0 -oslo.versionedobjects>=1.17.0 # Apache-2.0 -Paste # MIT +oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 +oslo.service>=1.24.0 # Apache-2.0 +oslo.versionedobjects>=1.28.0 # Apache-2.0 +Paste>=2.0.2 # MIT @@ -27,0 +30 @@ PasteDeploy>=1.5.0 # MIT +python-freezerclient>=1.3.0 # Apache-2.0 @@ -29,2 +32,2 @@ python-glanceclient>=2.8.0 # Apache-2.0 -python-novaclient>=9.0.0 # Apache-2.0 -python-cinderclient>=3.1.0 # Apache-2.0 +python-novaclient>=9.1.0 # Apache-2.0 +python-cinderclient>=3.2.0 # Apache-2.0 @@ -41 +44 @@ WebOb>=1.7.1 # MIT -oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 +oslo.i18n>=3.15.3 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 9ff128d..5c9f04a 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -6 +6 @@ hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 -botocore>=1.0.0 # Apache-2.0 +botocore>=1.5.1 # Apache-2.0 @@ -14,0 +15 @@ testtools>=1.4.0 # MIT +kubernetes>=1.0.0 # Apache-2.0 @@ -17,2 +18,3 @@ python-glanceclient>=2.8.0 # Apache-2.0 -python-novaclient>=9.0.0 # Apache-2.0 -python-cinderclient>=3.1.0 # Apache-2.0 +python-novaclient>=9.1.0 # Apache-2.0 +python-cinderclient>=3.2.0 # Apache-2.0 +python-freezerclient>=1.3.0 # Apache-2.0 @@ -22,3 +24,3 @@ python-troveclient>=2.2.0 # Apache-2.0 -reno!=2.3.1,>=1.8.0 # Apache-2.0 -openstackdocstheme>=1.16.0 # Apache-2.0 -os-api-ref>=1.0.0 # Apache-2.0 +reno>=2.5.0 # Apache-2.0 +openstackdocstheme>=1.17.0 # Apache-2.0 +os-api-ref>=1.4.0 # Apache-2.0