We are amped to announce the release of: zaqar 4.0.0: OpenStack Queuing and Notification Service This release is part of the ocata release series. Download the package from: https://tarballs.openstack.org/zaqar/ For more details, please see below. 4.0.0 ^^^^^ New Features * The OSprofiler is integrated to Zaqar in Ocata. It is a library from oslo. It aims to analyse the performance bottleneck issue by making possible to generate one trace per request affecting all involved services and build a tree of calls. * A new queue action is added so that users can purge a queue quickly. That means all the messages and subscriptions will be deleted automatically but the metadata of the queue will be kept. * Add migration support for Zaqar's sqlalchemy storage driver. * This feature is the third part of subscription confirmation feature. Support to send email to subscriber if confirmation is needed. To use this feature, user need to set the config option "external_confirmation_url", "subscription_confirmation_email_template" and "unsubscribe_confirmation_email_template". The confirmation page url that will be used in email subscription confirmation before notification, this page is not hosted in Zaqar server, user should build their own web service to provide this web page. The subscription_confirmation_email_template let user to customize the subscription confimation email content, including topic, body and sender. The unsubscribe_confirmation_email_template let user to customize the unsubscribe confimation email content, including topic, body and sender too. Changes in zaqar 3.0.0.0rc1..4.0.0 ---------------------------------- f196f9a Subscription Confirmation Support-3 5a6823a Update .gitreview for stable/ocata 1ddd9ca Fix sqlalchemy migration 90dedee Enable swift in the gate hook 83bbd7b Updated from global requirements a7dfeaf Switch to decorators.idempotent_id 8477f95 Add validation of limit value in querying flavors and poolss ef7110c Add the release note for OSprofiler feature 95b2290 Remove py34 support in favour of py35 dfaa49b Make the log work when deploy Zaqar with uwsgi 8e32ee2 Fix wrong value of require_confirmation 8d9be75 Replace assertTrue(*>*) with assertGreater 4a40387 Use _LW() to wrap the warning message 1a9760d Fix typos in HACKING.rst 40083a7 Swift storage e22ce5e Fix unit test unstable issue 455cf6e don't allow get/update/delete subscirtions with wrong queue 53f0c9f Removes unnecessary utf-8 encoding 874fd85 Updated from global requirements 39452c9 Use pop in message_delete_many 828bab9 Enable coverage report in console output 649ae23 Add tempest test for purge queue 908d75c Update README.rst ca1501a Fix CI failure b41802c Add API ref of purging queue 77af509 insert Apache 2.0 license into source file 5b1f915 Support purge queue -- websocket d91cfb7 Clean transport logs for API endpoints fb8da5d Add support for sqlalchemy migration based on alembic 460c345 Support purge queue -- wsgi 5a901c2 Convert user getting started guide to rst effbf89 Add OSprofiler guide 3851144 Updated from global requirements 75c0279 Updated from global requirements 03b3c6d Show team and repo badges on README 4a1a6b3 Remove startup check from devstack 65a6953 Use KEYSTONE_IDENTITY_BACKEND e5eee70 Consistent with the project code 35deeef Fix SSL and verbose issue of zaqar bench 4732417 Integrate OSprofiler with Zaqar 308934a Fix an typo in flavors.inc 7fb030e Fix zaqar client gate 00fc1f6 Use build interval and timeout from conf aa0889a Fix zaqar-ui clone issue a6ceea7 Add Feature section in doc 160f958 Reword description of grace 5edc11d Add missing policy check for queue stats and share c19d0a0 Don't send confirmation notification if it's not enabled 1032436 Add some descriptions for resources in API Ref de075b8 Updated from global requirements 2fae4ec Update .coveragerc after the removal of openstack directory 9d9147a Enanble Zaqar-UI in devstack f3b4745 Fix doc build if git is absent 4f8470e Enable release notes translation 69df431 Updated from global requirements 885435b Fix a typo: remove redundant 'the' f0d216b Change assertTrue(isinstance()) by optimal assert 670afbf Subscription Confirmation Support-2 176a3e5 Update reno for stable/newton d7c16b9 Don't check key format in mongodb f31a6e2 [api-ref] Remove temporary block in conf.py 993e804 Using oslo_config's min attribute b5e3980 Add install guide for ubuntu and suse Diffstat (except docs and test files) ------------------------------------- .coveragerc | 2 +- .gitreview | 1 + HACKING.rst | 2 +- README.rst | 13 +- api-ref/source/claims.inc | 6 +- api-ref/source/conf.py | 38 +- api-ref/source/flavors.inc | 2 +- api-ref/source/health.inc | 3 + api-ref/source/messages.inc | 6 + api-ref/source/parameters.yaml | 10 +- api-ref/source/pools.inc | 8 +- api-ref/source/queues.inc | 46 +- api-ref/source/samples/purge-queue-request.json | 3 + api-ref/source/subscription.inc | 4 + bench-requirements.txt | 4 +- devstack/gate/gate_hook.sh | 7 +- devstack/plugin.sh | 41 +- devstack/settings | 5 + .../os-zaqar-apiGettingStarted.xml | 1153 -------------------- etc/oslo-config-generator/zaqar.conf | 3 + etc/policy.json.sample | 2 + install-guide/source/install-obs.rst | 545 +++++++++ install-guide/source/install-rdo.rst | 48 +- install-guide/source/install-ubuntu.rst | 529 +++++++++ install-guide/source/install.rst | 2 + ...ate-OSprofiler-with-zaqar-59d0dc3d0326947d.yaml | 6 + .../notes/purge-queue-6788a249ee59d55a.yaml | 4 + .../sqlalchemy-migration-6b4eaebb6e02a449.yaml | 3 + ...onfirmation-support-email-0c2a56cfedc5d1e2.yaml | 15 + releasenotes/source/conf.py | 4 +- releasenotes/source/index.rst | 1 + releasenotes/source/newton.rst | 6 + requirements.txt | 21 +- samples/html/confirmation_web_service_sample.py | 86 ++ samples/html/subscriptionConfirmation.html | 148 +++ samples/html/unsubscriptionConfirmation.html | 145 +++ setup.cfg | 7 +- test-requirements.txt | 16 +- tools/test-setup.sh | 33 + tox.ini | 3 +- zaqar/api/v2/endpoints.py | 53 +- zaqar/api/v2/request.py | 19 + zaqar/bench/conductor.py | 8 +- zaqar/bench/config.py | 3 + zaqar/bench/consumer.py | 2 +- zaqar/bench/helpers.py | 23 +- zaqar/bench/observer.py | 2 +- zaqar/bench/producer.py | 2 +- zaqar/bootstrap.py | 13 + zaqar/common/configs.py | 48 +- zaqar/common/consts.py | 2 + zaqar/notification/notifier.py | 28 +- zaqar/notification/tasks/mailto.py | 78 +- zaqar/profile.py | 116 ++ zaqar/storage/base.py | 31 + zaqar/storage/mongodb/driver.py | 60 +- zaqar/storage/mongodb/messages.py | 26 +- zaqar/storage/mongodb/options.py | 2 +- zaqar/storage/mongodb/subscriptions.py | 14 +- zaqar/storage/mongodb/utils.py | 2 - zaqar/storage/pipeline.py | 15 +- zaqar/storage/pooling.py | 36 +- zaqar/storage/redis/driver.py | 31 +- zaqar/storage/redis/models.py | 9 +- zaqar/storage/redis/subscriptions.py | 55 +- zaqar/storage/redis/utils.py | 2 +- zaqar/storage/sqlalchemy/driver.py | 44 +- zaqar/storage/sqlalchemy/migration/__init__.py | 0 zaqar/storage/sqlalchemy/migration/alembic.ini | 54 + .../migration/alembic_migrations/README.md | 73 ++ .../sqlalchemy/migration/alembic_migrations/env.py | 96 ++ .../migration/alembic_migrations/script.py.mako | 34 + .../alembic_migrations/versions/001_liberty.py | 72 ++ .../alembic_migrations/versions/002_placeholder.py | 30 + .../alembic_migrations/versions/003_placeholder.py | 30 + .../alembic_migrations/versions/004_placeholder.py | 30 + .../alembic_migrations/versions/005_placeholder.py | 30 + zaqar/storage/sqlalchemy/migration/cli.py | 118 ++ zaqar/storage/swift/__init__.py | 0 zaqar/storage/swift/claims.py | 188 ++++ zaqar/storage/swift/controllers.py | 23 + zaqar/storage/swift/driver.py | 78 ++ zaqar/storage/swift/messages.py | 352 ++++++ zaqar/storage/swift/options.py | 31 + zaqar/storage/swift/queues.py | 124 +++ zaqar/storage/swift/subscriptions.py | 159 +++ zaqar/storage/swift/utils.py | 166 +++ zaqar/storage/utils.py | 10 +- .../services/messaging/json/messaging_client.py | 10 + .../unit/storage/sqlalchemy_migration/__init__.py | 0 .../sqlalchemy_migration/test_db_manage_cli.py | 89 ++ .../sqlalchemy_migration/test_migrations.py | 173 +++ .../sqlalchemy_migration/test_migrations_base.py | 194 ++++ .../unit/transport/websocket/v2/test_claims.py | 6 +- .../unit/transport/websocket/v2/test_messages.py | 3 +- .../transport/websocket/v2/test_queue_lifecycle.py | 53 + .../unit/transport/wsgi/v2_0/test_subscriptions.py | 4 + .../unit/transport/wsgi/v2_0/test_validation.py | 22 + zaqar/transport/validation.py | 55 +- zaqar/transport/wsgi/app.py | 3 + zaqar/transport/wsgi/driver.py | 7 +- zaqar/transport/wsgi/v2_0/__init__.py | 13 +- zaqar/transport/wsgi/v2_0/flavors.py | 16 +- zaqar/transport/wsgi/v2_0/health.py | 2 + zaqar/transport/wsgi/v2_0/homedoc.py | 13 + zaqar/transport/wsgi/v2_0/ping.py | 3 + zaqar/transport/wsgi/v2_0/pools.py | 15 +- zaqar/transport/wsgi/v2_0/purge.py | 82 ++ zaqar/transport/wsgi/v2_0/queues.py | 9 +- zaqar/transport/wsgi/v2_0/stats.py | 4 + zaqar/transport/wsgi/v2_0/subscriptions.py | 39 +- zaqar/transport/wsgi/v2_0/urls.py | 4 + 146 files changed, 6616 insertions(+), 1647 deletions(-) Requirements updates -------------------- diff --git a/bench-requirements.txt b/bench-requirements.txt index 5cc80af..2026a02 100644 --- a/bench-requirements.txt +++ b/bench-requirements.txt @@ -3,2 +3,2 @@ marktime>=0.2.0 -python-zaqarclient>=0.0.2 -os-client-config!=1.6.2,>=1.4.0 \ No newline at end of file +python-zaqarclient>=1.1.0 +os-client-config>=1.13.1 # Apache-2.0 \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 5b913db..361ce53 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4 +4 @@ -pbr>=1.6 # Apache-2.0 +pbr>=1.8 # Apache-2.0 @@ -5,0 +6 @@ pbr>=1.6 # Apache-2.0 +alembic>=0.8.10 # MIT @@ -10 +11 @@ iso8601>=0.1.11 # MIT -keystonemiddleware!=4.1.0,!=4.5.0,>=4.0.0 # Apache-2.0 +keystonemiddleware>=4.12.0 # Apache-2.0 @@ -13,2 +14,2 @@ python-memcached>=1.56 # PSF -WebOb>=1.2.3 # MIT -stevedore>=1.16.0 # Apache-2.0 +WebOb>=1.6.0 # MIT +stevedore>=1.17.1 # Apache-2.0 @@ -17 +18 @@ oslo.cache>=1.5.0 # Apache-2.0 -oslo.config>=3.14.0 # Apache-2.0 +oslo.config!=3.18.0,>=3.14.0 # Apache-2.0 @@ -20 +21,2 @@ oslo.i18n>=2.1.0 # Apache-2.0 -oslo.log>=1.14.0 # Apache-2.0 +oslo.log>=3.11.0 # Apache-2.0 +oslo.messaging>=5.14.0 # Apache-2.0 @@ -22,2 +24,3 @@ oslo.serialization>=1.10.0 # Apache-2.0 -oslo.utils>=3.16.0 # Apache-2.0 -oslo.policy>=1.9.0 # Apache-2.0 +oslo.utils>=3.18.0 # Apache-2.0 +oslo.policy>=1.17.0 # Apache-2.0 +osprofiler>=1.4.0 # Apache-2.0 @@ -28 +31 @@ autobahn>=0.10.1 # MIT License -requests>=2.10.0 # Apache-2.0 +requests!=2.12.2,>=2.10.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 93fe258..30aa62c 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -12,0 +13 @@ pymongo!=3.1,>=3.0.2 # Apache-2.0 +python-swiftclient>=3.2.0 # Apache-2.0 @@ -13,0 +15 @@ websocket-client>=0.32.0 # LGPLv2+ +PyMySQL>=0.7.6 # MIT License @@ -16 +18 @@ websocket-client>=0.32.0 # LGPLv2+ -coverage>=3.6 # Apache-2.0 +coverage>=4.0 # Apache-2.0 @@ -19,0 +22 @@ python-subunit>=0.0.18 # Apache-2.0/BSD +testscenarios>=0.4 # Apache-2.0/BSD @@ -21,0 +25,2 @@ testtools>=1.4.0 # MIT +oslo.db>=4.15.0 # Apache-2.0 +testresources>=0.2.4 # Apache-2.0/BSD @@ -24,2 +29,2 @@ testtools>=1.4.0 # MIT -sphinx!=1.3b1,<1.3,>=1.2.1 # BSD -oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0 +sphinx!=1.3b1,<1.4,>=1.2.1 # BSD +oslosphinx>=4.7.0 # Apache-2.0 @@ -28 +33 @@ oslotest>=1.10.0 # Apache-2.0 -reno>=1.8.0 # Apache2 +reno>=1.8.0 # Apache-2.0 @@ -32,0 +38,3 @@ tempest>=12.1.0 # Apache-2.0 + +#OSprofiler +osprofiler>=1.4.0 # Apache-2.0