Open Stack

Hi Chen,

>1.       resize the master node. Using command like “nova resize”, to add
more cpu & memory and other resources for this single instance.
Instance reside is not supported on Sahara level. But you can always do
that on nova level. Sahara doesn't use any specific information about
flavor (except disk mappings: swap, etc.). How Hadoop reacts on hardware
change fully depends on distribution used.

However if you need to resize scalable processes (e.g. datanode) you can
create new instance with larger flavor and delete old one. Sahara supports
such operation out of the box. Of course, processes that live in one copy
can't be scaled in such manner.

>2.       Split processes on the master node to several nodes.
As in previous item this can be done only for scalable processes. Each
Sahara plugin defines which processes are scalable and which are not.

>Another example, I already have a big cluster, and now I want to enable a
new service on it.
>1.       I would like to start a new node for the new service and add the
new node into my cluster.
You can easily do that. Cluster scale in Sahara is not limited with
nodegroups it already has. You can add any new nodegroups to the cluster.

>2.       Or, just start the new service on a node which already in the
cluster.
This is not supported. But you can start node with joined old and new
services on it and delete old node. As I said before this could be done
only for scalable processes.

>Currently, Sahara start cluster and manage all nodes based on “templates”,
so everything on “living” has to be pre-defined.
This is not true. You can add whatever you like as new nodegroups.

>So, my question here is:
>     Is it possible for Sahara to do things like that ?
Sahara supports scaling by adding and removing nodes. It doesn't support
changing of existing nodes.

>     Would Sahara want to support things like this ?
Changing existing nodes totally makes sense (like modifications of heat
stack), but looks a bit comprehensive. Plugins will need to know on how to
change existing services. But for hadoop distributions with managers
(Ambari in HDP, CMC in Cloudera) it should be easy.

>     If yes, any plans in the past, in the future?
>     If not, any special reasons ??
As I remember there were no such plans because you could always get desired
configuration by adding/removing nodes. If you need master nodes upgrade
you can always create new cluster and migrate data to it.

Thanks,
Andrew.

On Sun, Mar 22, 2015 at 7:03 PM, Li, Chen <chen.li at intel.com> wrote:

>  Hi Sahara,
>
>
>
> Recently, I learned Sahara support nodes scale up & scale down, which
> means “scale” has been limited to node number.
>
> Is it possible to do a “instance scale”?
>
>
>
> For example, I build a small cluster at first, several slave nodes
> (running datanode & nodemanager) and a single master node (running all
> other processes)
>
> I keep increasing the number of slave nodes, and in a moment, my master
> node has become the performance bottleneck of the whole cluster.
>
>
>
> In this case, I would like to do several things, such as:
>
> 1.       resize the master node. Using command like “nova resize”, to add
> more cpu & memory and other resources for this single instance.
>
> 2.       Split processes on the master node to several nodes.
>
>
>
> I think it make sense to users.
>
> Or the whole “performance bottleneck” staff would never happen in real
> world  ???
>
>
>
> Another example, I already have a big cluster, and now I want to enable a
> new service on it.
>
> 1.       I would like to start a new node for the new service and add the
> new node into my cluster.
>
> 2.       Or, just start the new service on a node which already in the
> cluster.
>
>
>
> Currently, Sahara start cluster and manage all nodes based on “templates”,
> so everything on “living” has to be pre-defined.
>
> Things on above breaks the whole “pre-defined” thing.
>
>
>
> So, my question here is:
>
>      Is it possible for Sahara to do things like that ?
>
>
>
>      Would Sahara want to support things like this ?
>
>      If yes, any plans in the past, in the future?
>
>      If not, any special reasons ??
>
>
>
>
>
> Looking forward to your reply.
>
>
>
> Thanks.
>
> -chen
>
>
>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150323/129ad0a7/attachment.html>