We are thrilled to announce the release of: glance 16.0.0: OpenStack Image Service This release is part of the queens release series. Download the package from: https://tarballs.openstack.org/glance/ For more details, please see below. 16.0.0 ^^^^^^ Prelude ******* * A plugin framework for customizing the processing of imported images before they become active is introduced in this release, along with a new plugin that injects image metadata properties into imported images. Known Issues ************ * The Pike release notes pointed out that although support had been added to run Glance as a WSGI application hosted by a web server, the Glance team recommended that Glance be run in its normal standalone configuration, particularly in production environments. We renew that recommendation for the Queens release. In particular, Glance tasks (which are required for the interoperable image import functionality) do not execute when Glance is run under uWSGI (which is the OpenStack recommended way to run WSGI applications hosted by a web server). This is in addition to the chunked transfer encoding problems addressed by Bug 1703856 and will be more difficult to fix. (Additionally, as far as we are aware, the fix for Bug 1703856 has never been tested at scale.) Briefly, Glance tasks are run by the API service and would have to be split out into a different service so that API alone would run under uWSGI. The Glance project team did not have sufficient testing and development resources during the Queens cycle to attempt this (or even to discuss whether this is in fact a good idea). The Glance project team is committed to the stability of Glance. As part of OpenStack, we are committed to The Four Opens. If the ability to run Glance under uWSGI is important to you, feel free to participate in the Glance community to help coordinate and drive such an effort. (We gently remind you that "participation" includes providing testing and development resources.) (https://bugs.launchpad.net/glance/+bug/1703856) (https://governance.openstack.org/tc/reference/opens.html) Upgrade Notes ************* * The default value for the API configuration option "workers" was previously the number of CPUs available. It has been changed to be the min of {number of CPUs, 8}. Any value set for that option, of course, is honored. See Bug 1748916 for details. * Some configuration is required in order to make the Interoperable Image Import functionality work correctly. In particular, the "node_staging_uri" value in the glance-api.conf file must be set. See the section on Interoperable Image Import in the Glance Administration Guide for more information. Deprecation Notes ***************** * With the introduction of the "web-download" import method, we consider the Image Service v2 API to have reached feature parity with the DEPRECATED v1 API in all important respects. Support for the Image Service API v1 ends with the Queens release. The v1 API was deprecated in Newton and will be removed from the codebase at the beginning of the Rocky development cycle. Please plan appropriately. (http://git.openstack.org/cgit/openstack/glance/commit/?id=63e6dbb 1eb006758fbcf7cae83e1d2eacf46b4ab) Security Issues *************** * The "web-download" import-method, intended to be a replacement for the popular Image Service API v1 "copy-from" functionality, is configurable so that you can avoid the vulnerabilty described in OSSN-0078. See the Interoperable Image Import section of the Glance Administration Guide for details. (https://wiki.openstack.org/wiki/OSSN/OSSN-0078) Bug Fixes ********* * The following are some highlights of the bug fixes included in this release. * Bug 1714240: Avoid restarting a child when terminating * Bug 1719252: Metadefs: Fix 500 for name with more than 80 chars * Bug 1720354: Correctly send auth request to oslo.policy * Bug 1733813: Fix 500 from image-import on queued images * Bug 1688189: Fix member create to handle unicode characters * Bug 1737952: Fix 500 if custom property name is greater than 255 * Bug 1744824: Fix py27 eventlet issue <0.22.0 * Bug 1748916: Glance default workers total overkill for modern servers * Bug 1749297: Fix 500 from list-tasks call with postgresql (https://code.launchpad.net/bugs/1749297) (https://code.launchpad.net/bugs/1748916) (https://code.launchpad.net/bugs/1744824) (https://code.launchpad.net/bugs/1737952) (https://code.launchpad.net/bugs/1688189) (https://code.launchpad.net/bugs/1733813) (https://code.launchpad.net/bugs/1720354) (https://code.launchpad.net/bugs/1719252) (https://code.launchpad.net/bugs/1714240) Other Notes *********** * The Interoperable Image Import section of the Image Service API v2 Reference Guide was updated to include the new "web-download" import method. * The section on Interoperable Image Import in the Glance Administration Guide has been updated. Please see that section of the Guide for information about the configuration required to make the import functionality work correctly. * The Database Management sections of the Glance Administration Guide have been revised and updated. This includes information about the current experimental status of rolling upgrades and zero- downtime database upgrades. (https://developer.openstack.org/api-ref/image/v2/) (https://docs.openstack.org/glance/queens/admin/index.html) Changes in glance 15.0.0.0rc1..16.0.0 ------------------------------------- ceb8b9a Regenerate sample config files 7d9fb6a Add Queens release note 12e79d6 Revise help text for uri filtering options 6cb77a8 Triggers shouldn't be execute in offline migration 280d98b Revise database rolling upgrade documentation 3ae4e44 api-ref: update interoperable image import info 1a3beb2 Fix config group not found error 0e9401f Migration support for postgresql ff75e08 Add validation to check if E-M-C is already in sync 33ca96a Revise interoperable image import documentation 625a469 Imported Translations from Zanata 1d672be Update Queens info about Glance and uWSGI a0e7b3b Imported Translations from Zanata da47f42 Update admin docs for web-download import method 51f4750 URI filtering for web-download d07e95a Make the Image status transition early a5327bc Cleanup basic import tasks 5501f90 Use bool instead of int for boolean filter value ef841b6 Limit default workers to 8 47cfc05 Offline migration support for postgresql 235713e Use configured value for import-methods header aef3b84 Imported Translations from Zanata cbd6c08 Fix bad usage of extend in list_image_import_opts cfb76d9 Update UPPER_CONSTRAINTS_FILE for stable/queens be7455d Update .gitreview for stable/queens d027102 Update Queens metadefs release note 32f8724 Update api-ref for v.2.6 3207481 Add release note for API v2.6 f91c3a7 Align Vers Neg Middleware to current API 08d1c7f Implementation of db check command 824badc Decouple Image Import Plugin Opts 49a1a0a Revise import property injection plugin releasenote 20fcf18 Correct 1-character typo 26a0978 Release note for Queens metadefs changes 7aaa4fc Regenerate sample configuration files 8682876 Exiting with user friendly message and SystemExit() f268df1 Modify glance manage db sync to use EMC 5704ba6 Add img_linked_clone to compute vmware metadefs daa3c88 Handle TZ change in iso8601 >=0.1.12 6e82ea0 Replace xml defusedxml fe6af4f Replace base functional tearDown with addCleanup 63e9548 Add functional test gates 9363e0b Skip one functional test 6f63d28 Fix py27 eventlet issue <0.22.0 c1930d4 Fix pip install failure 64eda44 Execute py35 functional tests under py35 environment 955b917 Enable Image Import per default and make current 223f2cf Adds 'web-download' import method d0f3735 Updated from global requirements 5517049 Skip one functional test ee597c4 Use addOnException to capture server logs on failure 466c104 Separate out functional tests 806adc5 Update Signature Documentation 97dbfb0 Add doc8 to pep8 check for glance project f5ae04b Implementation of Inject metadata properties 6e39734 Resolve unit test failures with going to oslo.serialization 2.3.0 631add1 Updated from global requirements ec950dc Updated from global requirements dbf7bbc Add documentation for image import plugins fe21b1d Update scrubber documentation d886d6d Scrubber refactor 1231fb5 Add hooks for Image Import plugins 6d31337 Updated from global requirements 8dba796 Fix 500 if custom property name is greater than 255 05e9bdb Fix member create to handle unicode characters af7b564 [import-tests] adds tests for image-import/staging 3ac9fa6 Updated from global requirements 830526c [import-tests] Enhance image import tests 96334ad Add fixture to only emit DeprecationWarning once 89feef0 Move 'upload_image' policy check to the controller 7d00507 Fix 500 from duplicate stage call f6c484b Updated from global requirements 778c467 Prevent image become active without disk and container formats d8bcf8b Fix 500 on ValueError during image-import 4c4663b Update the documentation links f8a5a40 Update the valid disk bus list for qemu and kvm hypervisors c1a845d Add the list of hw_version supported by vmware driver. 0988244 Updated from global requirements d48c841 Utilize LimitingReader for staging data 579a0f9 Fix 500 from image-import on 'active' image 7a67c43 Fix 500 from stage call on non-existing image ef5742c Fix unstage after staging store denies write 02cce55 Updated from global requirements 7eb2fcc Delete data if image is deleted after staging call a19fdb0 Fix 500 from image-import on queued images a919fa1 Use new oslo.context arg names 0d95efd Use new oslo.db base test case 4a2626c Fix the wrong URL e078c7e Correct related section for enable_image_import 1531af4 Fix SQLAlchemy reference link f914e37 Remove setting of version/release from releasenotes 9177957 Updated from global requirements 4720cc0 Fix format of configuration/configuring.rst db4a83f Removing unreachable line from stage() method a345384 Wrong description in ImageMembersController.update dc32afe Updated from global requirements 2da4cd8 Updated from global requirements 914bf76 Correct sphinx syntax of glance doc e621958 Update http deploy docs to be a bit more explicit 9e9743f Clarify log message 14d209f Updated from global requirements d5eed8d Document new URL format 8026e7f Update api-ref about 403 for image location changes 3134ee0 Make ImageTarget behave like a dictionary 53ec4d5 Document Glance Registry deprecation bf7887a Replace body_file with class to call uwsgi.chunked_read() e571e18 tests: replace .testr.conf with .stestr.conf f557b54 Deprecate Registry and it's config opts e6769ab Update spec-lite info in contributors' docs 2562da2 Fix 500 if user passes name with more than 80 characters 1c13d8a Remove use of deprecated optparse module e34d50a Replace DbMigrationError with DBMigrationError 6c75371 Clean up api-ref index page ff4974d Updated from global requirements 184b27d Fix a typo in swift_store_utils.py: replace Vaid with Valid 3e55487 TrivialFix: Fix wrong test case 45eeaa7 Revert "Remove team:diverse-affiliation from tags" 824daf5 Update image statuses doc for latest change bf24d5e Update Rally Job related files b0b102b Add default configuration files to data_files 7dfaa55 Switch base to latest in link address 2428801 Align default policy in code with the one in conf 2f8edee Fix missing some content of glance database creation. 6df72f9 Updated from global requirements 86d4a76 Updated from global requirements 62ee9e7 Clean up database section of admin docs d304d2c Add image import docs to admin guide 1041c1f Updated from global requirements 877cd16 Avoid restarting a child when terminating 97158c3 Open Queens for data migrations e132129 Change variable used by log message 7101d9c api-ref: add 'protected' query filter 60248f0 Update invalid links of User doc 480722a Separate module reference from contributor/index page 7151016 Updated from global requirements 471fd8d Optimize the way to serach file 'glance-api-paste.ini' 4366a74 Fix api_image_import tasks stuck in 'pending' 1705d3d Alembic should use oslo_db facades d38c8d2 Correct group name in config 8005ee9 api-ref: add interoperable image import docs 2efc5b8 Add release note for Glance Pike RC-2 d3871a5 Fix Image API 'versions' response 917d0d0 Updated from global requirements 692dc04 Return 404 for import-info call b90ad25 Add 'tasks_api_access' policy b6e4dda Add 'api_image_import' type to task(s) schemas 8886dcd Update invalid path and link for Image Properties e17a349 Fix 500 error from image-stage call d4917cf Fix 500 error from image-import call 72cd126 Imported Translations from Zanata e2d42db Update reno for stable/pike c80d032 Add the missing i18n import 08cd24d api-ref: update container_format, disk_format 545fff5 Update URL home-page in documents according to document migration 8db99e2 Add a local bindep.txt override 8b4f955 Properly validate metadef objects 98d17a6 Changes behaviour when an image fails uploading Diffstat (except docs and test files) ------------------------------------- .gitignore | 3 +- .gitreview | 1 + .stestr.conf | 3 + .testr.conf | 8 - .zuul.yaml | 9 + HACKING.rst | 7 +- README.rst | 1 + api-ref/source/heading-level-guide.txt | 31 + api-ref/source/index.rst | 4 +- api-ref/source/v2/images-images-v2.inc | 6 + api-ref/source/v2/images-import.inc | 288 + .../source/v2/images-parameters-descriptions.inc | 24 +- api-ref/source/v2/images-parameters.yaml | 40 +- api-ref/source/v2/index.rst | 1 + .../source/v2/metadefs-namespaces-properties.inc | 2 +- api-ref/source/v2/metadefs-parameters.yaml | 11 +- .../v2/samples/image-import-g-d-request.json | 5 + .../v2/samples/image-import-w-d-request.json | 6 + .../v2/samples/image-info-import-response.json | 10 + .../samples/metadef-properties-list-response.json | 4 +- .../v2/samples/schemas-image-show-response.json | 4 +- .../v2/samples/schemas-images-list-response.json | 4 +- api-ref/source/versions/index.rst | 5 + .../versions/samples/image-versions-response.json | 4 +- api-ref/source/versions/versions.inc | 2 + bindep.txt | 22 + etc/glance-api.conf | 360 +- etc/glance-cache.conf | 229 +- etc/glance-image-import.conf.sample | 231 + etc/glance-manage.conf | 12 +- etc/glance-registry.conf | 185 +- etc/glance-scrubber.conf | 437 +- etc/metadefs/compute-libvirt-image.json | 4 +- etc/metadefs/compute-vmware-flavor.json | 12 +- etc/metadefs/compute-vmware.json | 5 + etc/oslo-config-generator/glance-image-import.conf | 4 + etc/policy.json | 9 +- etc/schema-image.json | 4 +- glance/api/policy.py | 35 +- glance/api/v2/discovery.py | 19 +- glance/api/v2/image_data.py | 53 +- glance/api/v2/image_members.py | 5 +- glance/api/v2/images.py | 93 +- glance/api/v2/metadef_namespaces.py | 2 +- glance/api/v2/metadef_objects.py | 7 +- glance/api/v2/metadef_properties.py | 7 +- glance/api/v2/metadef_tags.py | 2 +- glance/api/v2/tasks.py | 21 +- glance/api/versions.py | 4 +- glance/async/flows/_internal_plugins/__init__.py | 221 + .../async/flows/_internal_plugins/web_download.py | 127 + glance/async/flows/api_image_import.py | 150 +- glance/async/flows/base_import.py | 28 +- glance/async/flows/ovf_process.py | 4 +- glance/async/flows/plugins/__init__.py | 31 + .../async/flows/plugins/inject_image_metadata.py | 101 + glance/async/flows/plugins/no_op.py | 62 + glance/async/flows/plugins/plugin_opts.py | 34 + glance/async/taskflow_executor.py | 6 + glance/cmd/api.py | 6 + glance/cmd/cache_manage.py | 388 +- glance/cmd/manage.py | 167 +- glance/cmd/registry.py | 6 + glance/cmd/scrubber.py | 6 + glance/common/config.py | 40 +- glance/common/property_utils.py | 5 + glance/common/scripts/utils.py | 7 +- glance/common/swift_store_utils.py | 2 +- glance/common/timeutils.py | 3 +- glance/common/utils.py | 51 + glance/common/wsgi.py | 62 +- glance/common/wsgi_app.py | 16 +- glance/context.py | 8 +- glance/db/migration.py | 4 +- glance/db/simple/api.py | 1 + .../db/sqlalchemy/alembic_migrations/__init__.py | 22 +- .../alembic_migrations/data_migrations/__init__.py | 8 +- .../data_migrations/queens_migrate01_empty.py | 26 + glance/db/sqlalchemy/alembic_migrations/env.py | 13 +- .../ocata01_add_visibility_remove_is_public.py | 72 - .../ocata01_add_visibility_remove_is_public.sql | 162 - .../versions/ocata_contract01_drop_is_public.py | 23 +- .../versions/ocata_expand01_add_visibility.py | 4 +- .../versions/pike01_drop_artifacts_tables.py | 41 - .../versions/queens_contract01_empty.py | 25 + .../versions/queens_expand01_empty.py | 25 + glance/db/sqlalchemy/api.py | 7 +- glance/db/utils.py | 1 + glance/domain/__init__.py | 12 +- glance/locale/en_GB/LC_MESSAGES/glance.po | 5789 ++++++++++++++++++++ glance/locale/ja/LC_MESSAGES/glance.po | 288 +- glance/opts.py | 22 +- glance/registry/__init__.py | 16 + glance/registry/api/__init__.py | 3 + glance/registry/client/__init__.py | 48 + glance/scrubber.py | 142 +- .../db/migrations/test_ocata_contract01.py | 3 + .../db/migrations/test_ocata_migrate01.py | 12 +- .../flows/plugins/test_inject_image_metadata.py | 128 + httpd/glance-api-uwsgi.ini | 3 +- httpd/uwsgi-glance-api.conf | 1 + rally-jobs/glance.yaml | 57 +- rally-jobs/plugins/README.rst | 4 +- rally-jobs/plugins/plugin_sample.py | 89 - .../notes/api-2-6-current-9eeb83b7ecc0a562.yaml | 59 + .../bp-inject-image-metadata-0a08af539bcce7f2.yaml | 30 + ...g-1719252-name-validation-443a2e2a36be2cec.yaml | 9 + .../notes/deprecate-registry-ff286df90df793f0.yaml | 12 + ...t-lite-spec-db-sync-check-3e2e147aec0ae82b.yaml | 27 + releasenotes/notes/pike-rc-2-acc173005045e16a.yaml | 89 + .../queens-metadefs-changes-daf02bef18d049f4.yaml | 22 + .../notes/queens-release-b6a9f9882c794c24.yaml | 87 + .../queens-uwsgi-issues-4cee9e4fdf62c646.yaml | 31 + .../notes/scrubber-refactor-73ddbd61ebbf1e86.yaml | 8 + releasenotes/source/conf.py | 13 +- releasenotes/source/index.rst | 1 + releasenotes/source/pike.rst | 6 + requirements.txt | 45 +- setup.cfg | 21 +- test-requirements.txt | 25 +- tox.ini | 91 +- 196 files changed, 12638 insertions(+), 2783 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index b733c4a..59e976d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,0 +6 @@ pbr!=2.1.0,>=2.0.0 # Apache-2.0 +defusedxml>=0.5.0 # PSF @@ -16,5 +17,5 @@ alembic>=0.8.10 # MIT -httplib2>=0.7.5 # MIT -oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 -oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.context>=2.14.0 # Apache-2.0 -oslo.utils>=3.20.0 # Apache-2.0 +httplib2>=0.9.1 # MIT +oslo.config>=5.1.0 # Apache-2.0 +oslo.concurrency>=3.25.0 # Apache-2.0 +oslo.context>=2.19.2 # Apache-2.0 +oslo.utils>=3.33.0 # Apache-2.0 @@ -22,5 +23,5 @@ stevedore>=1.20.0 # Apache-2.0 -futurist!=0.15.0,>=0.11.0 # Apache-2.0 -taskflow>=2.7.0 # Apache-2.0 -keystoneauth1>=3.1.0 # Apache-2.0 -keystonemiddleware>=4.12.0 # Apache-2.0 -WSME>=0.8 # MIT +futurist>=1.2.0 # Apache-2.0 +taskflow>=2.16.0 # Apache-2.0 +keystoneauth1>=3.3.0 # Apache-2.0 +keystonemiddleware>=4.17.0 # Apache-2.0 +WSME>=0.8.0 # MIT @@ -30 +31 @@ PrettyTable<0.8,>=0.7.1 # BSD -Paste # MIT +Paste>=2.0.2 # MIT @@ -32 +33 @@ Paste # MIT -jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT +jsonschema<3.0.0,>=2.6.0 # MIT @@ -34 +35 @@ python-keystoneclient>=3.8.0 # Apache-2.0 -pyOpenSSL>=0.14 # Apache-2.0 +pyOpenSSL>=16.2.0 # Apache-2.0 @@ -36 +37 @@ pyOpenSSL>=0.14 # Apache-2.0 -six>=1.9.0 # MIT +six>=1.10.0 # MIT @@ -38,6 +39,6 @@ six>=1.9.0 # MIT -oslo.db>=4.24.0 # Apache-2.0 -oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 -oslo.log>=3.22.0 # Apache-2.0 -oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 -oslo.middleware>=3.27.0 # Apache-2.0 -oslo.policy>=1.23.0 # Apache-2.0 +oslo.db>=4.27.0 # Apache-2.0 +oslo.i18n>=3.15.3 # Apache-2.0 +oslo.log>=3.36.0 # Apache-2.0 +oslo.messaging>=5.29.0 # Apache-2.0 +oslo.middleware>=3.31.0 # Apache-2.0 +oslo.policy>=1.30.0 # Apache-2.0 @@ -53,2 +54,2 @@ debtcollector>=1.2.0 # Apache-2.0 -cryptography!=2.0,>=1.6 # BSD/Apache-2.0 -cursive>=0.1.2 # Apache-2.0 +cryptography!=2.0,>=1.9 # BSD/Apache-2.0 +cursive>=0.2.1 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index fcb8a43..fa3a515 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -16,2 +16,2 @@ fixtures>=3.0.0 # Apache-2.0/BSD -mock>=2.0 # BSD -sphinx>=1.6.2 # BSD +mock>=2.0.0 # BSD +sphinx!=1.6.6,>=1.6.2 # BSD @@ -20 +20 @@ testrepository>=0.0.18 # Apache-2.0/BSD -testresources>=0.2.4 # Apache-2.0/BSD +testresources>=2.0.0 # Apache-2.0/BSD @@ -22 +22 @@ testscenarios>=0.4 # Apache-2.0/BSD -testtools>=1.4.0 # MIT +testtools>=2.2.0 # MIT @@ -24,2 +24,3 @@ psutil>=3.2.2 # BSD -oslotest>=1.10.0 # Apache-2.0 -os-testr>=0.8.0 # Apache-2.0 +oslotest>=3.2.0 # Apache-2.0 +os-testr>=1.0.0 # Apache-2.0 +doc8>=0.6.0 # Apache-2.0 @@ -29 +30 @@ PyMySQL>=0.7.6 # MIT License -psycopg2>=2.5 # LGPL/ZPL +psycopg2>=2.6.2 # LGPL/ZPL @@ -31,2 +32,2 @@ pysendfile>=2.0.0 # MIT -qpid-python;python_version=='2.7' # Apache-2.0 -xattr>=0.4 # MIT +qpid-python>=0.26;python_version=='2.7' # Apache-2.0 +xattr>=0.9.2 # MIT @@ -36,3 +37,3 @@ python-swiftclient>=3.2.0 # Apache-2.0 -os-api-ref>=1.0.0 # Apache-2.0 -openstackdocstheme>=1.16.0 # Apache-2.0 -reno!=2.3.1,>=1.8.0 # Apache-2.0 +os-api-ref>=1.4.0 # Apache-2.0 +openstackdocstheme>=1.18.1 # Apache-2.0 +reno>=2.5.0 # Apache-2.0