We are happy to announce the release of: qinling 1.0.0: Function as a Service This release is part of the rocky release series. The source is available from: http://git.openstack.org/cgit/openstack/qinling Download the package from: https://tarballs.openstack.org/qinling/ For more details, please see below. Changes in qinling aee1b062bab33c7e6a5016a1b569bc2305010a41..1.0.0 ------------------------------------------------------------------ c67f84b Add release note for function alias 12fa919 Delete check of function alias e924577 Support to specify workload type for runtime b3c5bfd Improve connecting to existing k8s cluster guide af6da47 Add upgrade description to release note 38be74a Add a tempest test case to verify connection from k8s pod to outside 2616494 Support alias for webhook creation 662794b Kubernetes: disable inter-pods connections f76ffc9 Raise specific exceptions instead of tenacity.RetryError 6cc83e7 Support alias for job creation 613e542 Runtime: less strict for resource limiting f8c99ec Creating functions using OpenStack Swift 77fc9f1 Increase etcd3gw dependency version 69a61e5 Add scope information when getting session from Keystone cd92cff Support alias for execution creation 121a537 Fix function download content-type not set correctly 01d769f Support to get pool information for the runtime 6b45191 periodics: fix service of function version 0 not expiring issue 4a4a18f Set file size limit to 50M in the python rumtime image 30fcd00 python2-runtime: limit cglimit listen on localhost only bcdbfb6 Add a preliminary structure for admin security guide abba847 The original image link does not work 13fb206 Remove default function_version in webhook resource def 347363d config: update api_workers default value to match help message 779bcf7 Enable oslo_config mutable configurations b7534dd Function Alias API: delete and update bbfe83e Invoke Python function in a directory rather than a zip package 16f5bb2 Add an example for function cookbook 18f8b9d Function Alias API: create and get 2b2a62d Add basic structure for user guide 7ced7b8 add release notes to README.rst d85799a Add install guide on ubuntu 16.04 a8425dc Add simple tempest testcases for resource limiting 3d06637 Fix typo 28f4440 Improve contributing page 180e1c6 Add release note for customized memory and cpu resources feature 2045799 devstack: use cgroupfs driver for both docker and k8s on Cent OS e886447 Runtime implementation for non-image type function. 670a7be fix tox python3 overrides e567904 replace file() with open() for python 3 compatibility b63d05d Function aliases: db scheme change 290fc69 Fix trivial errors in log formatting 958b2d2 Store error info in execution result when failed e5ee23d Delete package only when the package is updated 045c94a Specify the runtime image version explicitly 3afca01 Improve doc 284fe74 rest_utils: simplify filter extracting 3aae175 Function Aliases spec proposal 2237daf Delete underlying resources when deleting function version 2a1301d Replace Chinese quotes with English quotes 5a4292e Fix typo 4a27a7a Improve the quick start guide a little bit f69f33c [Trivialfix]Modify a grammatical error 92d96f5 Improve the setup scripts to support multi-nics e61e18c Fix function package downloading 40a89a3 Fix incompatible requirements 2edca38 kubernetes: add V1DeleteOptions when deleting services 33b4337 Improve qinling doc 8757a10 Fix the permission issue in devstack plugin ff38280 Runtime implementation for image type function 61dc02f Document how to make Qinling work with existing k8s cluster e22faa6 Outline for Qinling doc 706948a Add release note for function versioning feature 4c085db Functional test for deleting function version 19cd85e Functional test for function version detach/get 0e8d91c Fix missing fields in API response 49ea542 Add function_id in function version API response 5d8f0c0 Functional test for creating job/webhook with function version 6ee16ce Functional test for creating execution with function version 2be2b43 Functional test for creating function version 08ded02 Function version API: scale_up/scale_down/detach/get_workers 3f303eb Delete all its versions when deleting function e0e8f3d Add cpu and memory_size options support for function c3a081f Support function version for webhook 7d79bf3 Support function version for job 710b613 Replace assertRaisesRegexp with assertRaisesRegex 885ed28 Support version for execution creation d4567df Upgrade db version to add cpu column for function 917132a Function versioning API: delete e8e8084 Function versioning API: get 9005cea Function versioning API: list 12d2ee6 Function versioning API: create 8331c23 Delete old package when updating function 674b839 Fix devstack plugin script 39a5a05 Function versioning: db scheme support 546d03e Include md5 in function package path 5572617 Depress the debug log from futurist.periodics ef5268e Add missing release note for the k8s certs change 7c09206 Fix typo: extention -> extension 4826a01 engine: set execution status upon exceptions 76d01bb Allow qinling to connect to k8s API with certificates 9c47eec spec doc for adding customized mem/cpu for function 1ea7e91 Remove force value of get_url in playbook 2865210 Function versioning spec proposal e6c4303 Switch qinling-tempest to a voting check and gate job bd29ab0 periodics: switch to use futurist 740bcd8 engine: delete workers from etcd after removing 0eef3c8 Install same version for k8s CLIs and k8s control plane services 8218068 Updated from global requirements 64d2b90 Updated from global requirements 5735423 add lower-constraints job f56c67f Add specs folder for Qinling 62a2790 add missing unit tests for methods of function controller d11172e Updated from global requirements 9bed24e Exclude tests and db migrations from coverage report c9c1c1a Add a basic feature description about Sync/Async 9f91d6f zuul: create Neutron config dir for the devstack job 2ad25a3 tempest plugin: save package in temporary dir 823b5b4 Updated from global requirements 7ba51a7 Make tempest check use devstack-tempest parent 5fdd96a Updated from global requirements 261dd0c tests: add more unit tests for qinling.orchestrator 20d10bc tests: add unit tests for qinling.orchestrator 0dd033d Updated from global requirements e865f19 kubeadm: make it also work on CentOS 7 6bec49c Updated from global requirements 2bef99e Improve the net_resolv_pre_kube function in network.sh 9922c20 Remove the resource limit for pod in k8s deployment 6c6bca6 Add Node.js runtime support ffc1ceb Add unzip functionality in sidecar e9eaef8 kubeadm: fix issue about obtaining k8s config 70e8458 Don't run tempest jobs when patching unit test and releasenote 23d1259 Follow the new PTI for document build 803bb7b orchestrator: fix several issues of kubernetes manager ec36b22 Replace uuid.uuid4() with uuidutils.generate_uuid() b0553c6 Updated from global requirements 05588c1 Add sidecar support for runtime b0d724a Update qinling doc location to docs.openstack.org f9b342b tests: add filesystem storage unit tests 5934149 tests: add default engine unit tests 5615dd5 Use uWSGI for python runtime 22d172b Fix potential KeyError in execution tests dabfa1a Use kubeadm for devstack 5215953 doc: use storyboard link in CONTRIBUTING 7fd9070 filesystem storage: don't ensure base path when initialization e587213 Don't run tempest jobs when patching document c424780 Update the admin guide doc structure 2651652 Fix link for keystonemiddleware page 462561c Fix minor typos in chinese spelling 9523466 Update mysql connection in doc d3356c3 Update home-page url 44e9752 Add coverage test script cea8723 tests: improve unit tests of runtime API controller 2826144 Fix the OpenStack-Helm page does not exist e3c5e70 engine: fix a typo in default_engine log message 78be927 Use storyboard link in Qinling doc e17b388 Improve doc 3f99e1b Zuul: Remove project name 9e217f9 kube.sh: don't download kubectl again when it exists 3372e12 kube.sh: fix wrong systemd mount flag 113a29b tox.ini: add missing comments for H204 and H205 399363b kube.sh: skip get-docker when Docker is installed from distro repo ad4a3df Fix a typo in quick start guide a229b4f Make shell script graceful 53416f7 Fix one doc error in developer guide 7250094 Zuul: Remove project name 3170f38 Fix a potential error about script file Diffstat (except docs and test files) ------------------------------------- .coveragerc | 3 + .zuul.yaml | 74 +- CONTRIBUTING.rst | 28 +- README.rst | 7 +- devstack/plugin.sh | 65 +- devstack/settings | 8 +- .../development-environment-devstack.rst | 24 +- etc/policy.json.sample | 5 + .../functions/python/openstack/create_thumbnail.py | 36 +- example/kubernetes/cfssl-ca-config.json | 17 + example/kubernetes/cfssl-client-csr.json | 7 + example/kubernetes/k8s_qinling_role.yaml | 77 ++ lower-constraints.txt | 131 +++ playbooks/legacy/qinling-dsvm-tempest/post.yaml | 80 -- playbooks/legacy/qinling-dsvm-tempest/run.yaml | 52 -- playbooks/post.yaml | 8 + playbooks/pre.yaml | 10 + qinling/api/controllers/v1/execution.py | 15 +- qinling/api/controllers/v1/function.py | 215 +++-- qinling/api/controllers/v1/function_alias.py | 156 ++++ qinling/api/controllers/v1/function_version.py | 361 ++++++++ qinling/api/controllers/v1/job.py | 26 +- qinling/api/controllers/v1/resources.py | 148 ++-- qinling/api/controllers/v1/root.py | 2 + qinling/api/controllers/v1/runtime.py | 50 +- qinling/api/controllers/v1/webhook.py | 43 +- qinling/cmd/api.py | 3 +- qinling/cmd/engine.py | 2 + qinling/config.py | 70 +- qinling/context.py | 2 +- qinling/db/api.py | 48 ++ qinling/db/sqlalchemy/api.py | 130 ++- .../versions/002_add_function_version_support.py | 69 ++ .../versions/003_add_cpu_column.py | 44 + .../versions/004_add_function_aliases_support.py | 54 ++ .../versions/005_add_trusted_for_runtime.py | 36 + qinling/db/sqlalchemy/model_base.py | 9 +- qinling/db/sqlalchemy/models.py | 76 +- qinling/engine/default_engine.py | 194 +++-- qinling/engine/utils.py | 71 +- qinling/exceptions.py | 15 + qinling/orchestrator/base.py | 17 +- qinling/orchestrator/kubernetes/manager.py | 272 +++++-- .../kubernetes/templates/deployment.j2 | 36 +- qinling/orchestrator/kubernetes/templates/pod.j2 | 11 +- .../orchestrator/kubernetes/templates/service.j2 | 3 +- qinling/orchestrator/kubernetes/utils.py | 14 +- qinling/rpc.py | 23 +- qinling/services/periodics.py | 86 +- qinling/storage/base.py | 50 +- qinling/storage/file_system.py | 137 +++- .../unit/api/controllers/v1/test_execution.py | 69 +- .../unit/api/controllers/v1/test_function_alias.py | 154 ++++ .../api/controllers/v1/test_function_version.py | 278 +++++++ .../api/controllers/v1/test_function_worker.py | 39 +- .../unit/orchestrator/kubernetes/test_manager.py | 904 +++++++++++++++++++++ qinling/utils/common.py | 19 +- qinling/utils/constants.py | 2 + qinling/utils/etcd_util.py | 47 +- qinling/utils/executions.py | 102 ++- qinling/utils/openstack/keystone.py | 2 + qinling/utils/openstack/swift.py | 16 +- qinling/utils/rest_utils.py | 47 +- qinling_tempest_plugin/config.py | 20 +- .../functions/nodejs/test_nodejs_async.js | 13 + .../functions/nodejs/test_nodejs_basic.js | 3 + .../functions/python/test_python_basic.py | 17 + .../functions/python/test_python_cpu_limit.py | 40 + .../functions/python/test_python_file_limit.py | 38 + .../functions/python/test_python_http_get.py | 24 + .../functions/python/test_python_memory_limit.py | 23 + .../python/test_python_positional_args.py | 17 + .../functions/python/test_python_process_limit.py | 41 + .../functions/python/test_python_sleep.py | 18 + qinling_tempest_plugin/functions/python_test.py | 17 - .../functions/test_python_file_limit.py | 38 - .../functions/test_python_positional_args.py | 17 - .../functions/test_python_process_limit.py | 43 - .../functions/test_python_sleep.py | 18 - qinling_tempest_plugin/plugin.py | 4 +- qinling_tempest_plugin/services/qinling_client.py | 199 ++++- ...ble-mutable-configuration-a0527660fa58532c.yaml | 6 + .../notes/function-aliasing-318abd48128b4084.yaml | 5 + .../function-versioning-6e23fc021c2a7c7e.yaml | 10 + ...-runtime-pool-information-440351400ad7f0b8.yaml | 5 + .../notes/isolate-k8s-pods-617fec5dc5fbd2d8.yaml | 12 + ...nling-k8s-apiserver-certs-1651e26de5ca001c.yaml | 21 + .../resources-customized-5f5382d40e375dc1.yaml | 10 + .../workload-type-support-d613cdb7bb90b2a2.yaml | 9 + releasenotes/source/index.rst | 2 +- releasenotes/source/pike.rst | 6 - releasenotes/source/queens.rst | 6 + requirements.txt | 22 +- runtimes/nodejs/Dockerfile | 14 + runtimes/nodejs/package-lock.json | 797 ++++++++++++++++++ runtimes/nodejs/package.json | 24 + runtimes/nodejs/server.js | 105 +++ runtimes/python2/Dockerfile | 10 +- runtimes/python2/README.md | 22 - runtimes/python2/cglimit.py | 124 +++ runtimes/python2/custom-entrypoint.sh | 18 +- runtimes/python2/requirements.txt | 1 + runtimes/python2/server.py | 151 ++-- runtimes/sidecar/Dockerfile | 23 + runtimes/sidecar/requirements.txt | 3 + runtimes/sidecar/sidecar.py | 104 +++ setup.cfg | 2 +- setup.py | 2 +- test-requirements.txt | 7 +- tools/cover.sh | 66 ++ tools/gate/dump_logs.sh | 97 --- tools/gate/funcs/common.sh | 55 -- tools/gate/funcs/kube.sh | 150 ---- tools/gate/funcs/network.sh | 88 -- tools/gate/kubeadm/funcs/common.sh | 60 ++ tools/gate/kubeadm/funcs/network.sh | 50 ++ tools/gate/kubeadm/playbook/deploy_k8s.yaml | 21 + .../playbook/roles/docker/defaults/main.yml | 2 + .../kubeadm/playbook/roles/docker/tasks/centos.yml | 26 + .../kubeadm/playbook/roles/docker/tasks/main.yml | 14 + .../kubeadm/playbook/roles/docker/tasks/ubuntu.yml | 22 + .../playbook/roles/k8s_cli/tasks/centos.yml | 36 + .../k8s_cli/tasks/install_kube_prompt_tasks.yml | 25 + .../kubeadm/playbook/roles/k8s_cli/tasks/main.yml | 16 + .../playbook/roles/k8s_cli/tasks/ubuntu.yml | 20 + .../playbook/roles/kube_master/tasks/main.yml | 46 ++ tools/gate/kubeadm/setup_gate.sh | 49 ++ tools/gate/kubeadm/vars.sh | 19 + tools/gate/kubeadm_aio.sh | 25 - tools/gate/setup_gate.sh | 41 - tools/gate/vars.sh | 42 - tools/kubeadm-aio/Dockerfile | 88 -- tools/kubeadm-aio/README.rst | 110 --- tools/kubeadm-aio/assets/etc/kube-cni | 2 - tools/kubeadm-aio/assets/etc/kube-role | 3 - tools/kubeadm-aio/assets/etc/kube-version | 3 - .../assets/etc/kubeadm-join-command-args | 1 - tools/kubeadm-aio/assets/etc/kubeadm.conf | 4 - tools/kubeadm-aio/assets/etc/kubeapi-device | 3 - tools/kubeadm-aio/assets/etc/kubelet-container | 3 - tools/kubeadm-aio/assets/kubeadm-aio | 54 -- .../assets/opt/cni-manifests/calico.yaml | 365 --------- .../assets/opt/cni-manifests/canal.yaml | 329 -------- .../assets/opt/cni-manifests/flannel.yaml | 94 --- .../assets/opt/cni-manifests/weave.yaml | 187 ----- .../assets/opt/nfs-provisioner/deployment.yaml | 73 -- .../assets/opt/nfs-provisioner/storageclass.yaml | 5 - tools/kubeadm-aio/assets/opt/rbac/dev.yaml | 15 - tools/kubeadm-aio/kubeadm-aio-launcher.sh | 94 --- tools/vagrant/Vagrantfile | 6 +- tools/vagrant/qinling.conf.sample | 4 +- tox.ini | 25 +- 208 files changed, 11143 insertions(+), 3179 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 3b6834a..c4d242a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -7 +7 @@ Babel!=2.4.0,>=2.3.4 # BSD -keystoneauth1>=3.3.0 # Apache-2.0 +keystoneauth1>=3.4.0 # Apache-2.0 @@ -9,2 +9,2 @@ keystonemiddleware>=4.17.0 # Apache-2.0 -oslo.concurrency>=3.20.0 # Apache-2.0 -oslo.config>=5.1.0 # Apache-2.0 +oslo.concurrency>=3.26.0 # Apache-2.0 +oslo.config>=5.2.0 # Apache-2.0 @@ -15 +15 @@ oslo.utils>=3.33.0 # Apache-2.0 -oslo.log>=3.30.0 # Apache-2.0 +oslo.log>=3.36.0 # Apache-2.0 @@ -19 +19 @@ pecan!=1.0.2,!=1.0.3,!=1.0.4,!=1.2,>=1.0.0 # BSD -setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,!=36.2.0,>=16.0 # PSF/ZPL +setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,!=36.2.0,>=21.0.0 # PSF/ZPL @@ -25,2 +25,2 @@ WSME>=0.8.0 # MIT -kubernetes>=4.0.0 # Apache-2.0 -PyYAML>=3.10 # MIT +kubernetes>=6.0.0 # Apache-2.0 +PyYAML>=3.12 # MIT @@ -29,2 +29,2 @@ croniter>=0.3.4 # MIT License -python-dateutil>=2.4.2 # BSD -tenacity>=3.2.1 # Apache-2.0 +python-dateutil>=2.5.3 # BSD +tenacity>=4.4.0 # Apache-2.0 @@ -32,2 +32,2 @@ PyMySQL>=0.7.6 # MIT License -etcd3gw>=0.2.0 # Apache-2.0 -cotyledon>=1.3.0 # Apache-2.0 +etcd3gw>=0.2.2 # Apache-2.0 +cotyledon>=1.3.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index b8f9530..73f7445 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -7,2 +7 @@ coverage!=4.4,>=4.0 # Apache-2.0 -sphinx>=1.6.2 # BSD -oslotest>=1.10.0 # Apache-2.0 +oslotest>=3.2.0 # Apache-2.0 @@ -14,2 +12,0 @@ futurist>=1.2.0 # Apache-2.0 -openstackdocstheme>=1.17.0 # Apache-2.0 -reno>=2.5.0 # Apache-2.0 @@ -17 +14 @@ reno>=2.5.0 # Apache-2.0 -kubernetes>=4.0.0 # Apache-2.0 +kubernetes>=6.0.0 # Apache-2.0