Open Stack

Hello, Stackers,

I'm pleased to share with you the progress we have made on the Senlin
project, which aims to be a flexible clustering/collection service for
OpenStack clouds. The project is now hosted on stackforge, with its
server code base[1] and a CLI client tool [2].


What Is Senlin
--------------

Senlin was designed to be a generic clustering/collection service to
manage collections of homogeneous objects exposed from other OpenStack
services. It was deeply rooted from Heat, which has already provided
certain level of support to the notions of resource groups and even
resource groups that can be auto-scaled. One of Senlin's design goals
is to complement Heat's orchestration service by offloading the auto-
scaling functionality from Heat so that Heat can be more focused on
orchestration itself while diverting feature requests/improvements to a
dedicated 'ASS-NG' (Auto-Scaling-Service Next-Generation), :)

Although the initial intent of Senlin was merely about an auto-scaling
service, we soon realized that what is really needed is about the
management of collections of homogenous objects. The auto-scaling, load-
balancing, placement, removal and even health checking are all policies
that users may and may not want to attach to such collections. These
policies would be customized for different cloud applications running on
an OpenStack cloud. Users may want their own policy implementations
easily incorporated and effected.

Further more, users may want to manage many groups of cloud objects, i.e.
build pools of such objects and manage them collectively. For example,
in the case of auto-scaling, it would be convenient to have a 'standby'
group to back a 'working' group so that when requests surge, objects can
be migrated from the 'standby' group to the 'working' group
instantenously.

Simply put, Senlin aims to provide an 'array' data type for programming
your OpenStack cloud. 


<TL;DR>


Senlin Concepts
---------------

There are some core concepts in Senlin's design which are created to
facilitate a flexible clustering service.

* Cluster
  A collection (group) of homogeneous objects (aka. Nodes).

* Node
  An object that can be a member of at most one cluster at any time.
  There can be 'orphan' nodes. Nodes are created by instantiating
  profiles.

* Profile
  A specification about the real resource to be managed. Senlin use this
  concept to make clusters agnostic to the varying nature of resources
  managed. A profile can be a Heat stack, a Nova server, a floating-IP
  and so on.

* Action
  Operations that can be performed on a cluster, a node, a profile or
  a policy. Actions are mostly executed in asynchronous manner. Senlin
  provides builtin actions, but there are possibilities to expose
  actions to users as customizable scripts that could be SSH executed
  on VM-style nodes.

* Policy
  A collection of rules to be enforced before and/or after an action is
  performed.

* Event
  Event records capture interesting activities happened as results of
  engine operations, policy checkings/violations etc. Senlin hopes that
  the event history will save users from digging into log files.

Note that although we only have one profile and a few policies in current
code base, we designed Senlin to be extendable in terms of profiles and
policies. We don't want the core 'cluster' and 'node' concept hijacked
by any particular profile and/or policy.


Current Status
--------------

After more than three months' work, Senlin is now taking shape -- surely 
not an ideal shape yet depending on the angle from which you are judging
it. With some basic functionalities in place, it is ready for a try-out.


How to Try It
-------------

Step 0: Clone and install the server and the client as you do with other
OpenStack services.

Step 1: Write a profile spec. Some examples are provided in the code
base. The only supported profile type is 'os.heat.stack' at present.

Step 2: Create a profile

  # senlin profile-create -t os.heat.stack -s mystack.spec myprofile

Step 3: Create a node or a cluster using the profile

  # senlin cluster-create -p myprofile -n 3 my-cluster

  # senlin node-create -p myprofile my-node

  # senlin node-create -p myprofile -c my-cluster another-node

Step 4+: Play with the profile, the cluster, the node

  # senlin profile-list
  # senlin profile-show myprofile | <ID> | <short-id>

  # senlin cluster-list
  # senlin cluster-show my-cluster | <ID> | <short-id>

  # senlin node-list
  # senlin node-show my-node | <ID> | <short-id>
  # senlin node-join -c my-cluster my-node
  # senlin node-leave my-node

  # senlin cluster-scale-out -n 1 my-cluster
  # senlin cluster-scale-in -n 1 my-cluster

  # senlin cluster-nodes-add -n my-node my-cluster
  # senlin cluster-nodes-del -n my-node my-cluster

Step n: create a policy and attach it to a cluster

  # senlin policy-create -t ScalingPolicy -s scale-policy sp01

  # senlin policy-list
  # senlin policy-show sp01 | <ID> | <short-id>

  # senlin cluster-policy-attach -p sp01 my-cluster

  # senlin cluster-policy-list my-cluster

  # senlin cluster-policy-enable -p sp01 my-cluster
  # senlin cluster-policy-disable -p sp01 my-cluster

  # senlin cluster-policy-detach -p sp01 my-cluster

Examine actions and events:

  # senlin action-list
  # senlin action-show <name> | <ID> | <short-id>

  # senlin event-list
  # senlin event-show <ID> | <short-id> ...


Helps Needed
------------

If you are reading this line, please consider help us making Senlin more
useful, more mature. We have a lot of topics that are meant to be a team
decision. We have a lot of known bugs to be fixed. We barely have any
useful documents for users. We know there are many more requests to be
considered and satisified.

Team uses the #senlin channle on freenode IRC, eagerly waiting for your
opinions, comments, critics, suggestions, proposals, ... whatever.


Frequently Asked Questions
--------------------------

Q: What does the word 'senlin' mean?
A: 'Senlin' is a Chinese Pinyin for 'forest'. We chose this codename in
   hope that 1) it won't be difficult for English speakers to pronounce;
   2) it at least is a metaphor for the work it's doing. It may have to
   be changed if there are trademark issues raised.

Q: What will be the impacts to Heat users?
A: Ideally, none. No disruptive changes to users are expected. When
   Senlin matures, some of Heat resource types could be re-implemented
   with a careful migration path during which no userland visible
   changes are anticipated, unless there are serious technical barriers
   forcing us to. Existing Heat templates that use ResourceGroup and/or
   AutoScalingGroups don't have to be changed, though the resource
   implementation may be invoking Senlin in the background.

Q: Senlin is more of a collection service than a clustering service in
   the traditional sense, true?
A: Yes. Senlin is about collections of homogeneous objects, rather than
   traditional HPC clusters, HA clusters etc. We use the term 'Cluster'
   in code just to avoid confusion with the builtin Python collections
   data type.


[1] http://git.openstack.org/cgit/stackforge/senlin
[2] http://git.openstack.org/cgit/stackforge/python-senlinclient