We are overjoyed to announce the release of: cloudkitty 9.0.0: Rating as a Service component for OpenStack This release is part of the stein release series. The source is available from: https://git.openstack.org/cgit/openstack/cloudkitty Download the package from: https://pypi.org/project/cloudkitty Please report issues through: https://bugs.launchpad.net/cloudkitty/+bugs For more details, please see below. 9.0.0 ^^^^^ New Features ************ * A v2 API has been bootstrapped. It is compatible with the v2 storage and will be the base for all upcoming API endpoints. It is marked as "EXPERIMENTAL" for now. Upgrade Notes ************* * The v1 API is now marked as "CURRENT". The API root is now built with Flask instead of pecan Security Issues *************** * The default policy for the "/v1/storage/dataframes" endpoint has been changed from "unprotected" (accessible by any unauthenticated used) to "admin_or_owner" (accessible only by admins or members of the project). Bug Fixes ********* * HashMap module field mapping matching has been fixed: Field mapping values are always stored as strings. However, metadatas to match can be floats or integers (eg vcpus or ram). Given that mappings were matched with "==" until now, integers or float metadatas did never match. Changes in cloudkitty 8.0.0..9.0.0 ---------------------------------- bae398d Bootstrap the v2 API 9ce6941 Update the default policy rule for /v1/storage/dataframes adbc025 Update the default metrics.yml file 4e7c16f Add storage backend documentation 1128962 Fix HashMap field mapping comparison 1cc1e8b Update admin documentation for Prometheus collector 2dac030 Update UPPER_CONSTRAINTS_FILE for stable/stein 815dc5d Update .gitreview for stable/stein 7bfe768 Change configuration schema and query process for Prometheus collector 38f694a Add HTTPS and auth support to Prometheus collector 6404ec1 Skip a cycle instead of retrying if a metric is not found in gnocchi 87ab314 Update the devstack plugin 087038e Fix InfluxDB storage total() method pagination be38ba7 Added details to storage state 6bf2c03 Fix gnocchi collector metadata collection 9c88e50 Add a custom JSONEncoder a9dbe07 Update the hashmap module documentation 89c8c73 Moved Rating module introduction to Rating index 347992d Add collector documentation e3171f8 Add some developer documentation for collectors 3f4f5bb Change the default storage backend to v2/influxdb c51dddf Add a tempest job running against python3 devstack b0487b6 Support upgrade checks 1ede03b Delete v2 gnocchi storage 2d7415a Remove the fake and meta collectors d23781b Make devstack-tempest job voting 626eea7 Added help to gnocchi collector options 6d1b329 Remove the fake fetcher 6eb9040 Remove the gnocchi transformer 838ec23 Add support for cafile option in the gnocchi fetcher e601f70 Add an introduction to the documentation e398932 Change log message when loading v2 storage 45f0a00 Made metric conf validation more generic 990a5ab Changed influxdb v2 storage option 'cacert' to 'cafile' 4b79447 Change the documentation layout 884977f Add missing help kwarg to oslo option in influx storage c4758e7 Adding an InfluxDB storage backend 6350923 Changed author email d28469e Make tenant_id a string in hashmap models de4a38e Convert legacy zuul jobs to new format afbc5cf Don't raise NoDataCollected in case of collect error 7881c15 Don't update the state of a scope when no data was collected 3118f40 Remove repetitions in hashmap.rst 641b501 Don't heartbeat manually in the cloudkitty orchestrator 7778986 Remove oslo_i18n.enable_lazy() 01f702c Enable python3.7 testing jobs aa0c15e Fixed Mapping Update API returned status 409 with duplicate project b3ae81d Adds doc8 check to pep8 d4eea25 Update fetcher options f55efdf Support pagination in v2 storage total() c417e5f Use global-requirements for requirements d55154a Add scope_id to orchestrator log fb0b176 Change configuration section names b47594b Don't quote {posargs} in tox.ini 0f79fbe Handle the scope id as a regular groupby attribute in storage d466190 Use python3 for documentation builds 5a1aed8 [Docs] Change the cli to fit the latest client c9c0503 Add support for cafile option in the gnocchi collector f1e7d3e Hard to read README from github because of wrong format 114d7eb Use openstack-tox-cover template aa5376a Update the formulation on OpenStack release for RDO in the documentation 0613bb3 Import legacy-cloudkitty-dsvm-install 2b695d8 Add a gnocchi fetcher 34dfad4 add python 3.6 unit test job 0e68005 switch documentation job to new PTI 145eb07 import zuul job settings from project-config fb58a31 Update reno for stable/rocky c5cd023 Removes unnecessary utf-8 encoding Diffstat (except docs and test files) ------------------------------------- .gitreview | 1 + .zuul.yaml | 65 +- HACKING.rst | 8 +- README.rst | 5 +- cloudkitty/api/app.py | 54 +- cloudkitty/api/root.py | 191 ++---- cloudkitty/api/v1/__init__.py | 57 ++ cloudkitty/api/{ => v1}/config.py | 2 +- cloudkitty/api/v1/controllers/report.py | 30 +- cloudkitty/api/v1/controllers/storage.py | 16 +- cloudkitty/api/{ => v1}/hooks.py | 0 cloudkitty/api/v2/__init__.py | 63 ++ cloudkitty/api/v2/example/__init__.py | 26 + cloudkitty/api/v2/example/example.py | 68 ++ cloudkitty/api/v2/utils.py | 240 +++++++ cloudkitty/cli/status.py | 53 ++ cloudkitty/collector/__init__.py | 39 +- cloudkitty/collector/exceptions.py | 19 + cloudkitty/collector/fake.py | 165 ----- cloudkitty/collector/gnocchi.py | 155 +++-- cloudkitty/collector/meta.py | 88 --- cloudkitty/collector/monasca.py | 33 +- cloudkitty/collector/prometheus.py | 183 +++-- cloudkitty/common/config.py | 34 +- cloudkitty/common/defaults.py | 5 +- cloudkitty/common/policies/__init__.py | 22 +- cloudkitty/common/policies/v1/__init__.py | 0 cloudkitty/common/policies/{ => v1}/collector.py | 0 cloudkitty/common/policies/{ => v1}/info.py | 0 cloudkitty/common/policies/{ => v1}/rating.py | 0 cloudkitty/common/policies/{ => v1}/report.py | 0 cloudkitty/common/policies/{ => v1}/storage.py | 2 +- cloudkitty/common/policies/v2/__init__.py | 0 cloudkitty/common/policies/v2/example.py | 36 + cloudkitty/fetcher/__init__.py | 8 +- cloudkitty/fetcher/fake.py | 44 -- cloudkitty/fetcher/gnocchi.py | 135 ++++ cloudkitty/fetcher/keystone.py | 57 +- cloudkitty/fetcher/source.py | 22 +- cloudkitty/i18n.py | 1 - cloudkitty/json_utils.py | 35 + cloudkitty/orchestrator.py | 25 +- cloudkitty/rating/hash/__init__.py | 3 +- cloudkitty/rating/hash/controllers/mapping.py | 2 + cloudkitty/rating/hash/controllers/threshold.py | 2 + cloudkitty/rating/hash/datamodels/mapping.py | 6 +- cloudkitty/rating/hash/datamodels/threshold.py | 6 +- ...91fd_update_tenant_id_type_from_uuid_to_text.py | 97 +++ cloudkitty/rating/hash/db/sqlalchemy/api.py | 16 + cloudkitty/rating/hash/db/sqlalchemy/models.py | 4 +- cloudkitty/state.py | 15 +- cloudkitty/storage/__init__.py | 22 +- cloudkitty/storage/v1/hybrid/__init__.py | 13 +- cloudkitty/storage/v1/hybrid/backends/gnocchi.py | 10 +- cloudkitty/storage/v1/sqlalchemy/__init__.py | 12 +- cloudkitty/storage/v1/sqlalchemy/models.py | 4 +- cloudkitty/storage/v2/__init__.py | 31 +- cloudkitty/storage/v2/gnocchi.py | 758 --------------------- cloudkitty/storage/v2/influx.py | 369 ++++++++++ cloudkitty/storage_state/__init__.py | 62 +- .../d9d103dd4dcf_add_state_management_columns.py | 37 + cloudkitty/storage_state/models.py | 10 +- .../gabbi/rating/hash/gabbits/hash-empty.yaml | 1 - .../gabbi/rating/hash/gabbits/hash-errors.yaml | 1 - .../gabbi/rating/hash/gabbits/hash-location.yaml | 1 - .../gabbi/rating/pyscripts/gabbits/pyscripts.yaml | 1 - cloudkitty/transformer/gnocchi.py | 82 --- cloudkitty/writer/osrf.py | 14 +- contrib/ci/csv_writer.py | 3 +- devstack/README.rst | 52 +- devstack/files/influxdb.conf | 571 ++++++++++++++++ devstack/plugin.sh | 55 +- devstack/settings | 15 +- .../{ => admin}/configuration/configuration.rst | 27 +- .../{ => admin}/configuration/samples/index.rst | 0 .../configuration/samples/policy-yaml.rst | 2 +- .../v1}/rating/hashmap.rst | 0 .../v1}/rating/pyscripts.rst | 0 .../v2/example/example_parameters.yml | 20 + .../api-reference/v2/example/http_status.yml | 1 + .../{admin => user}/rating/graph/hashmap.dot | 0 etc/cloudkitty/api_paste.ini | 6 +- etc/cloudkitty/metrics.yml | 33 +- lower-constraints.txt | 50 ++ playbooks/cloudkitty-tempest-full/post.yaml | 15 - playbooks/cloudkitty-tempest-full/run.yaml | 54 -- .../add-gnocchi-fetcher-b8a6e2ea49fcfec5.yaml | 5 + ...dd-influx-storage-backend-3ace5b451e789e64.yaml | 9 + .../notes/add-scope-key-58135c2a5c6dae68.yaml | 5 + .../notes/added-v2-api-1ef829355c2feea4.yaml | 11 + .../default-to-v2-storage-a5ecac7e73dafa6d.yaml | 11 + .../deprecate_section_name-9f1ce1f84d09adf8.yaml | 5 + ...occhi-metadata-collection-74665e862483a383.yaml | 7 + ...shmap-mapping-value-match-56570510203ce3e5.yaml | 7 + .../notes/fix-lock-release-74d112c8599c9a59.yaml | 5 + .../harden-dataframes-policy-7786286525e52dfb.yaml | 6 + .../remove-fake-fetcher-9c264520a3cec9d0.yaml | 4 + ...move-fake-meta-collectors-5ed94ab1165e9661.yaml | 4 + ...emove-gnocchi-transformer-1dad750b9ba6c2e4.yaml | 4 + ...remove-v2-gnocchi-storage-a83bd58008bfd92e.yaml | 5 + ...work-prometheus-collector-02bd6351d447e4fe.yaml | 6 + ...work-prometheus-collector-f9f34a3792888dad.yaml | 6 + ...iod-if-nonexistent-metric-ba56a671e68f5bf5.yaml | 6 + .../status-upgrade-check-fdcf054643e071d8.yaml | 5 + releasenotes/source/index.rst | 1 + releasenotes/source/rocky.rst | 6 + requirements.txt | 24 +- setup.cfg | 10 +- setup.py | 2 +- test-requirements.txt | 15 +- tox.ini | 22 +- 195 files changed, 5923 insertions(+), 3175 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 059174d..334eb42 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4 +4 @@ -pbr>=1.6 # Apache-2.0 +pbr>=2.0.0,!=2.1.0 # Apache-2.0 @@ -6,2 +6,2 @@ alembic>=0.8.0 # MIT -eventlet!=0.18.3,>=0.18.2 # MIT -keystonemiddleware!=4.1.0,>=4.0.0 # Apache-2.0 +eventlet>=0.18.2,!=0.18.3,!=0.20.1,!=0.21.0,!=0.23.0 # MIT +keystonemiddleware>=4.0.0 # Apache-2.0 @@ -9,2 +9,2 @@ gnocchiclient>=2.5.0 # Apache-2.0 -python-monascaclient>=1.7.0 # Apache-2.0 -python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0 +python-monascaclient>=1.9.0 # Apache-2.0 +python-keystoneclient>=1.9.0,!=2.1.0 # Apache-2.0 @@ -14 +14 @@ PasteDeploy>=1.5.0 # MIT -pecan>=1.0.0 # BSD +pecan>=1.0.0,!=1.0.2,!=1.0.3,!=1.0.4,!=1.2 # BSD @@ -16 +16 @@ WSME>=0.8 # MIT -oslo.config>=3.7.0 # Apache-2.0 +oslo.config>=3.7.0,!=4.3.0,!=4.4.0 # Apache-2.0 @@ -22 +22 @@ oslo.log>=1.14.0 # Apache-2.0 -oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 +oslo.messaging>=5.24.2,!=9.0.0 # Apache-2.0 @@ -26 +26,2 @@ oslo.utils>=3.5.0 # Apache-2.0 -SQLAlchemy<1.1.0,>=1.0.10 # MIT +oslo.upgradecheck>=0.1.1 # Apache-2.0 +SQLAlchemy>=1.0.10,!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8 # MIT @@ -30 +31,4 @@ tooz>=1.28.0 # Apache-2.0 -voluptuous>=0.11.1,<1.0.0 # BSD-3 +voluptuous>=0.11.1 # BSD License +influxdb>=5.1.0,!=5.2.0 # MIT +Flask>=1.0.2 # BSD +Flask-RESTful>=0.3.5 # BSD diff --git a/test-requirements.txt b/test-requirements.txt index 9943c25..4705526 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -6 +6 @@ -hacking>=0.12.0,!=0.13.0,<0.14 # Apache-2.0 +hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 @@ -8,2 +8,2 @@ hacking>=0.12.0,!=0.13.0,<0.14 # Apache-2.0 -coverage>=3.6 # Apache-2.0 -kombu<4.0.0 # BSD +coverage>=3.6,!=4.4 # Apache-2.0 +kombu>=4.0.0,!=4.0.2 # BSD @@ -15 +15 @@ mock>=1.2 # BSD -sphinx>=1.6.2 # BSD +sphinx>=1.6.2,!=1.6.6,!=1.6.7 # BSD @@ -18 +18 @@ oslotest>=1.10.0 # Apache-2.0 -sphinxcontrib-httpdomain # BSD +sphinxcontrib-httpdomain>=1.6.0 # BSD @@ -20 +20,4 @@ sphinxcontrib-pecanwsme>=0.8 # Apache-2.0 -reno>=1.8.0 # Apache2 +reno>=1.8.0 # Apache-2.0 +doc8>=0.6.0 # Apache-2.0 +Pygments>=2.2.0 # BSD license +os-api-ref>=1.0.0 # Apache-2.0