We contentedly announce the release of: storlets 13.0.0: Middleware and Compute Engine for an OpenStack Swift compute framework that runs compute within a Swift cluster This release is part of the caracal release series. The source is available from: https://opendev.org/openstack/storlets Download the package from: https://tarballs.openstack.org/storlets/ Please report issues through: https://bugs.launchpad.net/storlets/+bugs For more details, please see below. Changes in storlets 12.0.0..13.0.0 ---------------------------------- d585857 reno: Update master for unmaintained/xena 96e3e2a reno: Update master for unmaintained/wallaby 5f254dd Add release note for 13.0.0 release 3c7d0b7 Document PUT with X-Storlet-Extra-Resources 8401b6c Allow limiting maximum extra resources b75465a Refactor FD handling 639fee6 Decouple StorletData from StorletRequest/Response 43c038a gateway: Ensure required interfaces are implemented caae8c4 Split out general implementation for containers 207f80b Remove unused constant 16c2923 Add CentOS Stream 9 job 876b695 Use built-in subprocess.DEVNULL 464310c Remove dev(el) package from container image 13a505d Bump java dependencies 1315cf9 Pick up jdk version from environment 40031d0 reno: Update master for unmaintained/yoga cafb36b Fix redundant default posargs for unit tests 0ccfe55 Remove unused .unittests c9e35be Skip functional tests for change in .gitignore 77509ad Skip functional tests when README is updated 7c8a63a Update repository structure 1b5dd53 Update links in README 9d80bd2 Remove unused STORLETS_DOCKER_SWIFT_*_ID 7e3b21d Ensure '-' in storlet object metadata 88369fe Run codespell to fix typos c227b51 Refactor functional tests with broken python storlet b43dca8 Skip storlet execution for symlink at object-server 50a4d4f Refactor SLO tests 5818bc0 Add option to limit pids within a container f7caa97 Remove unnecessary client config file generation 4aaa9d3 Remove unused bin/ files 513f6f7 Refactor config file generation/modification 3ed0275 Generalize container related options ede1fe4 Allow binding storlet applications to specific CPUs or memory nodes ddf7d26 Limit cpu and memory used by each storlet container 4bc6c2c coveragerc: Remove non-existent path 60e75cf Drop redundant try-except block 70d3540 Add Python 3.11 to supported runtime dd612dd Remove redundant cast dc81cef Restrict access to storlet log file 5ad5880 Restrict access to local storlet/dependency files 8f17a11 Remove note about old pip's behavior 07ca6fc Use devstack functions to manage swift/keystone services a4a3948 Drop explicit install of uwsgi-plugin-python3 8a8baf6 Fix inconsistent indentations 4d6eac6 Remove deploy targets 15a1b77 Update python classifier in setup.cfg 8346ea1 Stop using deprecated APIs be62155 Drop redundant type casts a67a505 Add mailmap entries for Takashi Kajinami c6d63b4 Enable lint check during build 3d24130 Bump java dependencies 55a9739 Stop to use the __future__ module. a9ad3a9 Offload entry point and command from image bd42751 Avoid using multiple return code ef064bb Remove workaround for old docker package b8f18c1 Remove unused cluster_config.json c5585ee Fix python shebang aac7ad9 Update master for stable/2023.2 Diffstat (except docs and test files) ------------------------------------- .coveragerc | 1 - .mailmap | 2 + .zuul.yaml | 18 +- CHANGELOG | 4 +- README.rst | 62 +- StorletSamples/java/CompressStorlet/build.xml | 4 +- StorletSamples/java/CsvStorlet/build.xml | 4 +- .../src/org/openstack/storlet/csv/CSVStorlet.java | 2 +- .../src/org/openstack/storlet/csv/Utils.java | 4 +- StorletSamples/java/ExecDepStorlet/build.xml | 4 +- StorletSamples/java/HalfStorlet/build.xml | 4 +- .../org/openstack/storlet/half/HalfStorlet.java | 5 +- StorletSamples/java/IdentityStorlet/build.xml | 4 +- .../storlet/identity/IdentityStorlet.java | 9 +- StorletSamples/java/MultiInputStorlet/build.xml | 4 +- .../org/openstack/storlet/MultiInputStorlet.java | 7 +- .../java/PartitionsIdentityStorlet/build.xml | 4 +- StorletSamples/java/TestMetadataStorlet/build.xml | 1 + StorletSamples/java/TestStorlet/build.xml | 1 + StorletSamples/java/ThumbnailStorlet/build.xml | 1 + bin/init_container.sh | 31 - bin/sbus | 20 - bin/storlets-daemon | 21 - bin/storlets-daemon-factory | 21 - bindep.txt | 4 - build.xml | 25 - cluster_config.json-sample | 11 - devstack/plugin.sh | 186 ++--- devstack/swift_config.py | 250 ------- .../writing_and_deploying_python_storlets.rst | 6 +- etc/object-server.conf-sample | 2 +- etc/proxy-server.conf-sample | 2 +- etc/storlet-docker-gateway.conf-sample | 2 +- install_libs.sh | 10 +- playbooks/storlets-functional/post.yaml | 18 +- playbooks/storlets-functional/pre.yaml | 8 - releasenotes/notes/13_0_0-f24d257669f564aa.yaml | 49 ++ releasenotes/source/2023.2.rst | 6 + releasenotes/source/index.rst | 2 + releasenotes/source/wallaby.rst | 2 +- releasenotes/source/xena.rst | 2 +- releasenotes/source/yoga.rst | 2 +- requirements.txt | 2 +- s2aio.sh | 19 +- setup.cfg | 11 +- setup.py | 2 +- src/c/sbus/sbus.c | 6 +- src/java/SBus/build.xml | 17 +- .../src/main/org/openstack/storlet/sbus/SBus.java | 16 +- .../org/openstack/storlet/sbus/SBusBackend.java | 2 +- .../storlet/sbus/ServerSBusInDatagram.java | 2 +- src/java/SCommon/build.xml | 8 +- src/java/SDaemon/build.xml | 9 +- .../main/org/openstack/storlet/daemon/SDaemon.java | 6 +- .../storlet/daemon/SExecutionManager.java | 2 +- .../openstack/storlet/daemon/SExecutionTask.java | 2 +- src/java/build.xml | 16 +- storlets/agent/common/server.py | 6 +- storlets/agent/daemon/files.py | 2 +- storlets/agent/daemon/server.py | 2 +- storlets/agent/daemon_factory/server.py | 14 +- storlets/gateway/common/logger.py | 23 +- storlets/gateway/common/stob.py | 39 +- storlets/gateway/gateways/base.py | 17 +- storlets/gateway/gateways/container/__init__.py | 0 storlets/gateway/gateways/container/gateway.py | 357 ++++++++++ storlets/gateway/gateways/container/runtime.py | 741 +++++++++++++++++++ storlets/gateway/gateways/docker/__init__.py | 4 +- storlets/gateway/gateways/docker/gateway.py | 373 +--------- storlets/gateway/gateways/docker/runtime.py | 786 +-------------------- storlets/gateway/gateways/stub.py | 15 +- storlets/sbus/client/client.py | 2 +- storlets/swift_middleware/handlers/base.py | 40 +- storlets/swift_middleware/handlers/obj.py | 27 +- storlets/swift_middleware/handlers/proxy.py | 147 ++-- storlets/swift_middleware/storlet_handler.py | 4 +- storlets/tools/deploy_storlet.py | 2 +- storlets/tools/extensions/ipython.py | 5 +- storlets/tools/testtools.py | 2 +- storlets/tools/utils.py | 4 +- test-requirements.txt | 4 - .../java/test_partitions_identity_storlet.py | 2 +- .../gateway/gateways/container/test_gateway.py | 551 +++++++++++++++ .../gateway/gateways/container/test_runtime.py | 423 +++++++++++ .../unit/swift_middleware/test_storlet_handler.py | 2 +- tox.ini | 11 +- 117 files changed, 3058 insertions(+), 3120 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index eee67ed..65e4ff0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -6 +6 @@ stevedore>=1.16.0 # Apache-2.0 -docker +docker>=1.5.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 17cf463..11efe7d 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,4 +0,0 @@ -# The order of packages is significant, because pip processes them in the order -# of appearance. Changing the order has an impact on the overall integration -# process, which may cause wedges in the gate later. -