[openstack-dev] [all] Announcing Project: Senlin - Clustering/Collection Service for OpenStack
tengqim at linux.vnet.ibm.com
Thu Mar 26 14:23:36 UTC 2015
I'm pleased to share with you the progress we have made on the Senlin
project, which aims to be a flexible clustering/collection service for
OpenStack clouds. The project is now hosted on stackforge, with its
server code base and a CLI client tool .
What Is Senlin
Senlin was designed to be a generic clustering/collection service to
manage collections of homogeneous objects exposed from other OpenStack
services. It was deeply rooted from Heat, which has already provided
certain level of support to the notions of resource groups and even
resource groups that can be auto-scaled. One of Senlin's design goals
is to complement Heat's orchestration service by offloading the auto-
scaling functionality from Heat so that Heat can be more focused on
orchestration itself while diverting feature requests/improvements to a
dedicated 'ASS-NG' (Auto-Scaling-Service Next-Generation), :)
Although the initial intent of Senlin was merely about an auto-scaling
service, we soon realized that what is really needed is about the
management of collections of homogenous objects. The auto-scaling, load-
balancing, placement, removal and even health checking are all policies
that users may and may not want to attach to such collections. These
policies would be customized for different cloud applications running on
an OpenStack cloud. Users may want their own policy implementations
easily incorporated and effected.
Further more, users may want to manage many groups of cloud objects, i.e.
build pools of such objects and manage them collectively. For example,
in the case of auto-scaling, it would be convenient to have a 'standby'
group to back a 'working' group so that when requests surge, objects can
be migrated from the 'standby' group to the 'working' group
Simply put, Senlin aims to provide an 'array' data type for programming
your OpenStack cloud.
There are some core concepts in Senlin's design which are created to
facilitate a flexible clustering service.
A collection (group) of homogeneous objects (aka. Nodes).
An object that can be a member of at most one cluster at any time.
There can be 'orphan' nodes. Nodes are created by instantiating
A specification about the real resource to be managed. Senlin use this
concept to make clusters agnostic to the varying nature of resources
managed. A profile can be a Heat stack, a Nova server, a floating-IP
and so on.
Operations that can be performed on a cluster, a node, a profile or
a policy. Actions are mostly executed in asynchronous manner. Senlin
provides builtin actions, but there are possibilities to expose
actions to users as customizable scripts that could be SSH executed
on VM-style nodes.
A collection of rules to be enforced before and/or after an action is
Event records capture interesting activities happened as results of
engine operations, policy checkings/violations etc. Senlin hopes that
the event history will save users from digging into log files.
Note that although we only have one profile and a few policies in current
code base, we designed Senlin to be extendable in terms of profiles and
policies. We don't want the core 'cluster' and 'node' concept hijacked
by any particular profile and/or policy.
After more than three months' work, Senlin is now taking shape -- surely
not an ideal shape yet depending on the angle from which you are judging
it. With some basic functionalities in place, it is ready for a try-out.
How to Try It
Step 0: Clone and install the server and the client as you do with other
Step 1: Write a profile spec. Some examples are provided in the code
base. The only supported profile type is 'os.heat.stack' at present.
Step 2: Create a profile
# senlin profile-create -t os.heat.stack -s mystack.spec myprofile
Step 3: Create a node or a cluster using the profile
# senlin cluster-create -p myprofile -n 3 my-cluster
# senlin node-create -p myprofile my-node
# senlin node-create -p myprofile -c my-cluster another-node
Step 4+: Play with the profile, the cluster, the node
# senlin profile-list
# senlin profile-show myprofile | <ID> | <short-id>
# senlin cluster-list
# senlin cluster-show my-cluster | <ID> | <short-id>
# senlin node-list
# senlin node-show my-node | <ID> | <short-id>
# senlin node-join -c my-cluster my-node
# senlin node-leave my-node
# senlin cluster-scale-out -n 1 my-cluster
# senlin cluster-scale-in -n 1 my-cluster
# senlin cluster-nodes-add -n my-node my-cluster
# senlin cluster-nodes-del -n my-node my-cluster
Step n: create a policy and attach it to a cluster
# senlin policy-create -t ScalingPolicy -s scale-policy sp01
# senlin policy-list
# senlin policy-show sp01 | <ID> | <short-id>
# senlin cluster-policy-attach -p sp01 my-cluster
# senlin cluster-policy-list my-cluster
# senlin cluster-policy-enable -p sp01 my-cluster
# senlin cluster-policy-disable -p sp01 my-cluster
# senlin cluster-policy-detach -p sp01 my-cluster
Examine actions and events:
# senlin action-list
# senlin action-show <name> | <ID> | <short-id>
# senlin event-list
# senlin event-show <ID> | <short-id> ...
If you are reading this line, please consider help us making Senlin more
useful, more mature. We have a lot of topics that are meant to be a team
decision. We have a lot of known bugs to be fixed. We barely have any
useful documents for users. We know there are many more requests to be
considered and satisified.
Team uses the #senlin channle on freenode IRC, eagerly waiting for your
opinions, comments, critics, suggestions, proposals, ... whatever.
Frequently Asked Questions
Q: What does the word 'senlin' mean?
A: 'Senlin' is a Chinese Pinyin for 'forest'. We chose this codename in
hope that 1) it won't be difficult for English speakers to pronounce;
2) it at least is a metaphor for the work it's doing. It may have to
be changed if there are trademark issues raised.
Q: What will be the impacts to Heat users?
A: Ideally, none. No disruptive changes to users are expected. When
Senlin matures, some of Heat resource types could be re-implemented
with a careful migration path during which no userland visible
changes are anticipated, unless there are serious technical barriers
forcing us to. Existing Heat templates that use ResourceGroup and/or
AutoScalingGroups don't have to be changed, though the resource
implementation may be invoking Senlin in the background.
Q: Senlin is more of a collection service than a clustering service in
the traditional sense, true?
A: Yes. Senlin is about collections of homogeneous objects, rather than
traditional HPC clusters, HA clusters etc. We use the term 'Cluster'
in code just to avoid confusion with the builtin Python collections
More information about the OpenStack-dev