We are joyful to announce the release of: freezer 3.0.0: The OpenStack Backup Restore and Disaster Recovery as a Service Platform This release is part of the newton release series. For more details, please see below. Changes in freezer 2.0.0.0rc1..3.0.0 ------------------------------------ df4ff63 Improve abort mechanism to support same functionality as stable/mitaka 7a08c4a Use --config-file from oslo instead of --config 9e6c963 Remove deprecated code fe6e049 Update .gitreview for stable/newton 83eff55 Fix Cinder Backups 7658243 Fixed nova backup and restore 4c1e7f8 Update message for job start/stop/abort 45fa458 change directory structure for unit tests 6c50cc3 Fix error: snaphost-->snapshot 78e16c6 Updated from global requirements c4ea71a Enable parallel job execution for the scheduler to allow abort signals 6e77d80 TrivialFix: Remove logging import unused 80c631f Fix order of arguments in assertEqual eef70d5 Require a backup name for backup/restore/admin 6f1b051 Remove tar option "--overwrite" when extracting archives c332a2b can't output the debug log for freezer-scheduler 4277d4a Added Validation layer for freezer jobs f1ab13c failed to start freezer-scheduler 20cafd2 Updated from global requirements 364e5e5 Fixed the use of --insecure cli option db1c31a Fixed use os.path.join to concatenate default logging location 9c94d30 Freezer metadata part2 659de17 Fix typo in the file 9aeed12 Fix freezer-scheduler start_date and endtime 819a5c1 Fix restore exception handling 66d7a68 Fix restore nova for multiple networks 47ef07d Changes freezer-agent consistency switches to hyphens/dashes 26351b0 Fixing Freezer with config file but no trickle throws error 5c35892 Remove discover from test-requirements 6ea14d9 Updated from global requirements 4937b5a add cindernative-backup-id option when restore cinder 304e06d Change cinder backup to define full backup or incremental backup 1c9bba4 Fail on non-existing path to backup ef5f496 Revise test_metadata_checksum ee7c8ae Add high-level backup utilities to BaseFreezerTest 98c93e6 Fix checksum generation so that it handles binary data f967745 Removed remnant of git conflict in README.rst 230d20a freezer client report "Unauthorized" error 43f9936 Correct reraising of exception 9d46c8e Fix log-config-append does not work when passed as config file a89998d Remove white space between print and () 2da929d Updated from global requirements 00db2b0 Fix Freezer Agent Logging bf17542 Fix LVM mountpoint issues. 9aa34db Always collect freezer.log when tests fail 4d8a3ce Fix tar error handling arguments b3bc12b Fix restore tar error handling e0383a1 Add integration test to verify compression options 836f94c Fix devstack-dsvm failure reporting 7fa0e6a Fix bug with snapshot removal 196a93e Fix cStringIO usage 5c9acf9 Add Tempest tests to validate checksum usage 694c98e Fix tox.ini file f16a2ee Fix messed up readme 3cdfd8f Fix exception propagation 8b8c766 First scenario test 75f2c9d Fix gnu tar error handling 5d1f445 New Better & Well Formatted Docs with Sphinx b4d4c38 Improve Tempest tests 5adb7fa Fix Tempest tests acquiring credentials 62e2c2a Fix incompatibility with keystoneauth 2b07da3 Fixed freezer-agent is not using the correct log f41b613 Fix the failure of loading the instance of MongoDbMode 9cddf83 Delete Invalid variable c976514 Remove version from setup.cfg b16f8b1 Switch to keystoneauth 87b4ab6 list backups stored in API 1bf9658 Adds checksum before backup and after restore for consistency check. fb5329c Freezer metadata 137520e Restore cindernative-vol-id by increment ab41b42 Fix chunk_size for tar engine af5f2b1 Clean up README 1b83ba7 Refactor Freezer authentication with keystone 62bd85a Use tempest credentials rather than hardcoded values e54c80e Use uuid instead of time as path for test data 04de157 Remove auto_snap de0344b Allow freezer to use trickle installed system wide 8fd7908 Updated from global requirements 73750a0 Fix storage not swift to backup nova or cinder c9254cd Update README with linux requirements for freezer 4ffc74e fix some rst syntax error of code block 9f8ec0b Add More Freezer Tempest Tests b4450c0 Restore cinder native backup pass right params and call right api 285e292 Use volume_id and timestamp to generate container name c8d08cf Add Freezer Tempest Tests c62a9da Generate a full backup first time 305067a Change $USER to $STACK_USER d468c38 Updating README.rst 428682e Fixing Freezer-Agent help 8bcf617 Fix glance client creation 8a9f378 Adding the freezer logo to the README 4b91ce6 Recreation swift client for each chunk during swift incremental restore be17aa3 Allow the user to use percentage on lvm snapsize 367a225 Fix creation of jobs with stopped state fd5d6ba Updated from global requirements 9efa12f Added --overwrite parameter for removing all files from restore directory. Default value --overwrite=False 0973cf7 Use correct type for freezer arguments ecfa4f6 Fix typos a6b771f Provides more details in the help of --restore-from-date option. Explains the behavior with an example. 7c0f5bc Rename mode default to fs 03fd939 Modify the vars display_name to 'name' 219fb2b Start to introduce tempest tests. 9d99a02 freezer_conf_d is created automatically 34d2168 enable output of metadata to a file e1fa806 Modify the function configure_freezer_scheduler Diffstat (except docs and test files) ------------------------------------- .gitreview | 1 + .pylintrc | 4 +- CREDITS.rst | 7 +- INSTALL.rst | 6 +- README.rst | 903 +++++++++++++------- devstack/gate_hook.sh | 13 +- devstack/lib/freezer | 6 +- devstack/settings | 7 +- .../images/admin/Service_Architecture_02.png | Bin 0 -> 49928 bytes .../images/admin/freezer_agent_backup_api.png | Bin 0 -> 34065 bytes .../images/admin/freezer_scheduler_api_scale.png | Bin 0 -> 49841 bytes freezer/apiclient/backups.py | 13 +- freezer/apiclient/client.py | 23 +- freezer/apiclient/exceptions.py | 4 +- freezer/apiclient/sessions.py | 6 +- freezer/common/config.py | 380 ++++++--- freezer/engine/engine.py | 268 ++++-- freezer/engine/exceptions.py | 19 + freezer/engine/tar/tar_builders.py | 8 +- freezer/engine/tar/tar_engine.py | 162 ++-- freezer/job.py | 270 ++++-- freezer/lib/pep3143daemon/daemon.py | 4 +- freezer/main.py | 133 ++- freezer/mode/cinder.py | 37 + freezer/mode/default.py | 35 - freezer/mode/fs.py | 35 + freezer/mode/mongo.py | 12 +- freezer/mode/mysql.py | 4 +- freezer/mode/nova.py | 37 + freezer/mode/sqlserver.py | 15 +- freezer/openstack/backup.py | 63 +- freezer/openstack/openstack.py | 81 -- freezer/openstack/osclients.py | 475 +++++++---- freezer/openstack/restore.py | 109 ++- freezer/scheduler/arguments.py | 49 +- freezer/scheduler/daemon.py | 93 +-- freezer/scheduler/freezer_scheduler.py | 51 +- freezer/scheduler/scheduler_job.py | 176 ++-- freezer/scheduler/shell.py | 61 +- freezer/scheduler/utils.py | 19 + freezer/scheduler/win_daemon.py | 2 +- freezer/snapshot/lvm.py | 200 ++--- freezer/snapshot/snapshot.py | 13 +- freezer/snapshot/vss.py | 21 +- freezer/storage/base.py | 486 +++++------ freezer/storage/exceptions.py | 19 + freezer/storage/fslike.py | 114 +-- freezer/storage/local.py | 5 +- freezer/storage/multiple.py | 90 +- freezer/storage/physical.py | 112 +++ freezer/storage/ssh.py | 15 +- freezer/storage/swift.py | 201 ++--- .../freezer_tempest_plugin/services/__init__.py | 0 .../unit/apiclient/test_apiclient_exceptions.py | 25 + .../unit/apiclient/test_apiclient_registration.py | 111 +++ .../unit/apiclient/test_apiclient_sessions.py | 223 +++++ freezer/utils/bandwidth.py | 83 -- freezer/utils/checksum.py | 168 ++++ freezer/utils/config.py | 40 +- freezer/utils/streaming.py | 18 +- freezer/utils/utils.py | 121 ++- freezer/utils/validator.py | 34 - freezer/utils/winutils.py | 8 +- freezer_logo.jpg | Bin 0 -> 50204 bytes requirements.txt | 37 +- setup.cfg | 5 +- setup.py | 36 +- specs/job-backup.conf.example | 2 +- test-requirements.txt | 27 +- .../unit/apiclient/test_apiclient_registration.py | 111 --- tox.ini | 15 +- 167 files changed, 9233 insertions(+), 5064 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index adeefc3..63992b1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,6 @@ -astroid<1.4.0 # breaks pylint 1.4.4 -setuptools>=16.0 -pbr>=1.6 +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +astroid<1.4.0 # LGPLv2.1 # breaks pylint 1.4.4 +setuptools!=24.0.0,>=16.0 # PSF/ZPL +pbr>=1.6 # Apache-2.0 @@ -5,9 +8,9 @@ python-swiftclient>=2.2.0 # Apache-2.0 -python-keystoneclient>=1.6.0,!=1.8.0 -python-cinderclient>=1.3.1 -python-glanceclient>=1.2.0 # Apache-2.0 -python-novaclient>=2.29.0,!=2.33.0 # Apache-2.0 -python-openstackclient>=2.0.0 # Apache-2.0 -oslo.utils>=3.2.0 -oslo.i18n>=1.5.0 # Apache-2.0 -oslo.log>=1.14.0 -oslo.config>=3.2.0 # Apache-2.0 +python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0 +python-glanceclient!=2.4.0,>=2.3.0 # Apache-2.0 +python-novaclient!=2.33.0,>=2.29.0 # Apache-2.0 +python-openstackclient>=2.1.0 # Apache-2.0 +oslo.utils>=3.16.0 # Apache-2.0 +oslo.i18n>=2.1.0 # Apache-2.0 +oslo.log>=1.14.0 # Apache-2.0 +oslo.config>=3.14.0 # Apache-2.0 +keystoneauth1>=2.10.0 # Apache-2.0 @@ -15,3 +18,3 @@ oslo.config>=3.2.0 # Apache-2.0 -PyMySQL>=0.6.2 # MIT License -pymongo>=3.0.2 -paramiko>=1.13.0 +PyMySQL!=0.7.7,>=0.6.2 # MIT License +pymongo!=3.1,>=3.0.2 # Apache-2.0 +paramiko>=2.0 # LGPLv2.1+ @@ -21 +24,3 @@ six>=1.9.0 # MIT -apscheduler +apscheduler # MIT License + +psutil>=1.1.1,<2.0.0 # BSD diff --git a/test-requirements.txt b/test-requirements.txt index b830d53..d2ff1cf 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,11 +1,16 @@ -flake8>=2.2.4,<=2.4.1 -hacking>=0.10.2,<0.11 -coverage>=3.6 -discover -mock>=1.2 -pylint==1.4.5 # GNU GPL v2 -python-subunit>=0.0.18 -sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 # BSD -oslosphinx>=2.5.0,!=3.4.0 # Apache-2.0 -testrepository>=0.0.18 -testtools>=1.4.0 +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +flake8<2.6.0,>=2.5.4 # MIT +hacking<0.11,>=0.10.2 +coverage>=3.6 # Apache-2.0 +mock>=2.0 # BSD +pylint==1.4.5 # GPLv2 +python-subunit>=0.0.18 # Apache-2.0/BSD +sphinx!=1.3b1,<1.3,>=1.2.1 # BSD +oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0 +testrepository>=0.0.18 # Apache-2.0/BSD +testtools>=1.4.0 # MIT + +# Tempest Plugin +tempest-lib>=0.14.0 # Apache-2.0