[openstack-dev] [release] taskflow 0.8.0
Joshua Harlow
harlowja at outlook.com
Wed Mar 25 22:17:11 UTC 2015
We are stoked to announce the release of:
taskflow 0.8.0: Taskflow structured state management library.
For more details, please see the git log history below and:
http://launchpad.net/taskflow/+milestone/0.8.0
Please report issues through launchpad:
http://bugs.launchpad.net/taskflow/
Notable changes
----------------
* The default engine storage is now always using a in-memory backend
instead of just using local objects and caching results and data
in those. This will help make the engine storage backend more
uniform and require less customizations to use.
* Large refactoring of backend path 'like' implementations (zookeeper,
directory/file based and the in-memory backend).
* New ``EventTimeListener`` that records when state changes
happen (useful for gathering data about how long your workflows
take, and how long the tasks in them take).
* Support for python 3.3 has been removed (as it is no longer tested and
3.4 is the stable 3.x release that is being tested against).
* Jobboards now have a trash() method that can be used
to 'trash' jobs that should not be operated on (and can be
used to look at for later analysis or for deleting).
* Conductors now have entrypoints that should be used for
loading (instead of accessing the direct conductor
classes). The fetch() method of ``taskflow.conductors.backends``
should be used to retrieve your desired conductor object.
* A new map/reduce functor task which has many potential uses.
* Removal of ``taskflow.types.timing.StopWatch`` (it has now been
moved to the oslo.utils library where it can be shared to a
larger audience); a proxy object will still exist in
``taskflow.types.timing`` but using the new location in
``oslo.utils.timeutils`` should be preferred instead.
* Usage of new debtcollector library functions.
* Better WBE send/receive data validation (using jsonschema).
* Atom provides/requires/optional/save_as now use an ordered dict or
ordered set by default so that any specified ordering is capable of
being maintained.
Changes in taskflow 0.7.1..0.8.0
--------------------------------
NOTE: Skipping requirement commits...
cdcb248 Adding test to improve CaptureListener coverage
cd54139 Prefer posixpath to os.path
f815480 By default use a in memory backend (when none is provided)
dc2fdaf Allow using shallow copy instead of deep copy
7d79e52 Move to the newer debtcollector provided functions
6723a18 Move to using the oslo.utils stop watch
579500d Ensure thread-safety of persistence dir backend
658d1f9 Ensure we are really setup before being connected
765037d Ensure docstring on storage properties
f7b3c23 Expose the storage backend being used
da73113 Use iteration instead of list(s) when extracting scopes
dd33ec7 Use binary/encode decode helper routines in dir backend
6db6363 Rename memory backend filesystem -> fake filesystem
3579410 Always return scope walker instances from `fetch_scopes_for`
374d91d Give the GC a break
6671ce4 Use the class name instead of the TYPE property in __str__
ebe3d6d Just use the class name instead of TYPE constant
5d2fb53 Attempt to extract traceback from exception
ac2b1be Use compatible map and update map/reduce task docs
d1742b1 Update engine docs with new validation stage
a1f9321 Ensure we register & deregister conductor listeners
87d6f58 Rename attribute '_graph' to '_execution_graph'
7e052e0 Add a log statement pre-validation that dumps graph info
5209961 Have this example exit non-zero if incorrect results
bf5164e Use a collections.namedtuple for the request work unit
0e952e6 Some small wbe engine doc tweaks
c1f0200 Add newline to avoid sphinx warning
526f287 Allow passing 'many_handler' to fetch_all function
12c28bb Ensure event time listener is in listeners docs
8602118 Add a in-memory backend dumping example
0a2928f Added a map and a reduce task
6529c1e Restructure the in-memory node usage
f6e2074 Switch to non-namespaced module imports
5996c8f Allow the storage unit to use the right scoping strategy
94eb978 Just use the local conf variable
844c2c6 Put underscore in-front of alchemist helper
9f64c47 lazy loading for logbooks and flowdetails
646ca59 Allow backend connection config (via fetch) to be a string
ad133ad Add + use failure json schema validation
67f0f51 Use ordered[set/dict] to retain ordering
f0de22c Allow injected atom args to be persisted
8e62483 add _listeners_from_job method to Conductor base
b9b6576 update uses of TimingListener to DurationListener
7d3ae77 Added EventTimeListner to record when events occur
8c08869 added update_flow_metadata method to Storage class
1478f52 Retain nested causes where/when we can
1f4e596 Denote issue 17911 has been merged/accepted
3e8eb91 Persistence backend refactor
c22c62d Remove support for 3.3
00ab628 Writers can now claim a read lock in ReaderWriterLock
47c0269 Add another probabilistic rw-lock test
20fdbba Add + use read/write lock decorators
89087fd Add no double writers thread test
d302b52 Use condition context manager instead of acquire/release
213fddb Remove condition acquiring for read-only ops
1f108da Set a no-op functor when none is provided
b56e453 Ensure needed locks is used when reading/setting intention
fd4772c Specialize checking for overlaps
217c635 Use links instead of raw block quotes
0806ac1 Rename the timing listeners to duration listeners
9500aee Add a bookshelf developer section
26dc932 Ensure the thread bundle stops in last to first order
c738693 Add warning about transient arguments and worker-based-engines
0e2a567 Ensure ordered set is pickleable
db22de9 Add node removal/disassociate functions
847d87d Add a fully functional orderedset
bebc960 Make the worker banner template part of the worker class
e3879bd Use compilation helper objects
53af5d3 Allow node finding to not do a deep search
3c806b1 Add a frozen checking decorator
b598897 Tweak functor used to find flatteners/storage routines
6da46b7 Add specific scoping documentation
517fa60 add jobboard trash method
71c97e3 Provide more contextual information about invalid periodics
0a97fb9 Fix lookup scoping multi-match ordering
1c03996 Stick to one space after a period
7a1a467 Refactor parts of the periodic worker
a44bc5e Use oslo.utils encodeutils for encode/decode functions
1267a2a Bring over pretty_tox.sh from nova/heat/others
12d81c9 Tweak some of the types thread safety docstrings
035b69d Add pypi link badges
9b5293f Switch the note about process pool executor to warning
b7eb26c Chain exceptions correctly on py3.x
2a1ffb7 Remove WBE experimental documentation note
37b72ec Use the enum library for the retry strategy enumerations
c9c0505 Use debtcollector library to replace internal utility
28490e5 add get_flow_details and get_atom_details to all backends
3342009 Tweaks to atom documentation
06ffcee Update Flow::__str__
145599a Add todo note for kombu pull request
d412947 Move 'provides' and 'name' to instance attributes
eaad725 Allow loading conductors via entrypoints
d1b6776 Make the dispatcher handler be an actual type
23c83e0 Improve multilock class and its associated unit test
Diffstat (except docs and test files)
-------------------------------------
README.rst | 8 +
requirements-py2.txt | 14 +-
requirements-py3.txt | 14 +-
setup.cfg | 4 +-
taskflow/atom.py | 192 +++---
taskflow/conductors/backends/__init__.py | 45 ++
taskflow/conductors/backends/impl_blocking.py | 183 ++++++
taskflow/conductors/base.py | 32 +-
taskflow/conductors/single_threaded.py | 171 +-----
taskflow/engines/action_engine/actions/base.py | 3 +-
taskflow/engines/action_engine/actions/retry.py | 6 +-
taskflow/engines/action_engine/actions/task.py | 8 +-
taskflow/engines/action_engine/analyzer.py | 28 +-
taskflow/engines/action_engine/compiler.py | 217 ++++---
taskflow/engines/action_engine/completer.py | 4 +-
taskflow/engines/action_engine/engine.py | 69 ++-
taskflow/engines/action_engine/runtime.py | 33 +-
taskflow/engines/action_engine/scheduler.py | 4 +-
taskflow/engines/action_engine/scopes.py | 97 +--
taskflow/engines/base.py | 38 +-
taskflow/engines/helpers.py | 26 +-
taskflow/engines/worker_based/dispatcher.py | 37 +-
taskflow/engines/worker_based/engine.py | 5 +-
taskflow/engines/worker_based/executor.py | 7 +-
taskflow/engines/worker_based/protocol.py | 90 ++-
taskflow/engines/worker_based/server.py | 71 +--
taskflow/engines/worker_based/types.py | 7 +-
taskflow/engines/worker_based/worker.py | 71 ++-
taskflow/examples/dump_memory_backend.py | 83 +++
taskflow/examples/graph_flow.py | 25 +
taskflow/examples/timing_listener.py | 2 +-
taskflow/exceptions.py | 37 +-
taskflow/flow.py | 5 +-
taskflow/jobs/backends/impl_zookeeper.py | 39 +-
taskflow/jobs/base.py | 19 +
taskflow/listeners/base.py | 15 +-
taskflow/listeners/timing.py | 75 ++-
taskflow/patterns/graph_flow.py | 13 +-
taskflow/persistence/backends/__init__.py | 3 +
taskflow/persistence/backends/impl_dir.py | 412 +++----------
taskflow/persistence/backends/impl_memory.py | 374 ++++++------
taskflow/persistence/backends/impl_sqlalchemy.py | 79 ++-
taskflow/persistence/backends/impl_zookeeper.py | 367 ++---------
taskflow/persistence/backends/sqlalchemy/tables.py | 6 +-
taskflow/persistence/base.py | 24 +-
taskflow/persistence/path_based.py | 245 ++++++++
taskflow/retry.py | 48 +-
taskflow/storage.py | 678
+++++++++++++--------
taskflow/task.py | 90 +++
taskflow/types/failure.py | 141 ++++-
taskflow/types/fsm.py | 10 +-
taskflow/types/graph.py | 33 +-
taskflow/types/notifier.py | 10 +-
taskflow/types/periodic.py | 149 +++--
taskflow/types/sets.py | 141 +++++
taskflow/types/timing.py | 264 +-------
taskflow/types/tree.py | 62 +-
taskflow/utils/deprecation.py | 150 +----
taskflow/utils/kazoo_utils.py | 22 +-
taskflow/utils/kombu_utils.py | 9 +-
taskflow/utils/lock_utils.py | 158 +++--
taskflow/utils/misc.py | 175 +++++-
taskflow/utils/schema_utils.py | 34 ++
taskflow/utils/threading_utils.py | 50 +-
test-requirements.txt | 5 +-
tools/pretty_tox.sh | 17 +
tools/subunit_trace.py | 308 ++++++++++
107 files changed, 5154 insertions(+), 2863 deletions(-)
Requirements updates
--------------------
diff --git a/requirements-py2.txt b/requirements-py2.txt
index 083caec..23c0a2d 100644
--- a/requirements-py2.txt
+++ b/requirements-py2.txt
@@ -14 +14,4 @@ ordereddict
-six>=1.7.0
+six>=1.9.0
+
+# Enum library made for <= python 3.3
+enum34
@@ -20 +23 @@ networkx>=1.8
-stevedore>=1.1.0 # Apache-2.0
+stevedore>=1.3.0,<1.4.0 # Apache-2.0
@@ -29,2 +32,5 @@ jsonschema>=2.0.0,<3.0.0
-oslo.utils>=1.2.0 # Apache-2.0
-oslo.serialization>=1.2.0 # Apache-2.0
+oslo.utils>=1.4.0,<1.5.0 # Apache-2.0
+oslo.serialization>=1.4.0,<1.5.0 # Apache-2.0
+
+# For deprecation of things
+debtcollector>=0.3.0,<0.4.0 # Apache-2.0
diff --git a/requirements-py3.txt b/requirements-py3.txt
index b04fc0a..d67ad1a 100644
--- a/requirements-py3.txt
+++ b/requirements-py3.txt
@@ -11 +11,4 @@ pbr>=0.6,!=0.7,<1.0
-six>=1.7.0
+six>=1.9.0
+
+# Enum library made for <= python 3.3
+enum34
@@ -17 +20 @@ networkx>=1.8
-stevedore>=1.1.0 # Apache-2.0
+stevedore>=1.3.0,<1.4.0 # Apache-2.0
@@ -23,2 +26,5 @@ jsonschema>=2.0.0,<3.0.0
-oslo.utils>=1.2.0 # Apache-2.0
-oslo.serialization>=1.2.0 # Apache-2.0
+oslo.utils>=1.4.0,<1.5.0 # Apache-2.0
+oslo.serialization>=1.4.0,<1.5.0 # Apache-2.0
+
+# For deprecation of things
+debtcollector>=0.3.0,<0.4.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index dd4f36d..23d9ac3 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -6 +6 @@ hacking<0.11,>=0.10.0
-oslotest>=1.2.0 # Apache-2.0
+oslotest>=1.5.1,<1.6.0 # Apache-2.0
@@ -8,0 +9 @@ testtools>=0.9.36,!=1.2.0
+testscenarios>=0.4
@@ -32 +33 @@ sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3
-oslosphinx>=2.2.0 # Apache-2.0
+oslosphinx>=2.5.0,<2.6.0 # Apache-2.0
More information about the OpenStack-dev
mailing list