<div dir="ltr">Hi Chen,<div><br></div><div><div>>1. resize the master node. Using command like “nova resize”, to add more cpu & memory and other resources for this single instance.</div><div>Instance reside is not supported on Sahara level. But you can always do that on nova level. Sahara doesn't use any specific information about flavor (except disk mappings: swap, etc.). How Hadoop reacts on hardware change fully depends on distribution used.</div><div><br></div><div>However if you need to resize scalable processes (e.g. datanode) you can create new instance with larger flavor and delete old one. Sahara supports such operation out of the box. Of course, processes that live in one copy can't be scaled in such manner.</div><div><br></div><div>>2. Split processes on the master node to several nodes.</div></div><div>As in previous item this can be done only for scalable processes. Each Sahara plugin defines which processes are scalable and which are not.</div><div><br></div><div><div>>Another example, I already have a big cluster, and now I want to enable a new service on it.</div><div>>1. I would like to start a new node for the new service and add the new node into my cluster.</div><div>You can easily do that. Cluster scale in Sahara is not limited with nodegroups it already has. You can add any new nodegroups to the cluster.</div><div><br></div><div>>2. Or, just start the new service on a node which already in the cluster.</div></div><div>This is not supported. But you can start node with joined old and new services on it and delete old node. As I said before this could be done only for scalable processes.</div><div><br></div><div><div>>Currently, Sahara start cluster and manage all nodes based on “templates”, so everything on “living” has to be pre-defined.</div><div>This is not true. You can add whatever you like as new nodegroups.</div></div><div><br></div><div><div>>So, my question here is:</div><div>> Is it possible for Sahara to do things like that ?</div><div>Sahara supports scaling by adding and removing nodes. It doesn't support changing of existing nodes. </div><div> </div><div>> Would Sahara want to support things like this ?</div><div>Changing existing nodes totally makes sense (like modifications of heat stack), but looks a bit comprehensive. Plugins will need to know on how to change existing services. But for hadoop distributions with managers (Ambari in HDP, CMC in Cloudera) it should be easy.</div><div><br></div><div>> If yes, any plans in the past, in the future?<br></div><div>> If not, any special reasons ??</div><div>As I remember there were no such plans because you could always get desired configuration by adding/removing nodes. If you need master nodes upgrade you can always create new cluster and migrate data to it.</div><div><br></div><div>Thanks,</div><div>Andrew.</div><div><br></div></div><div><div class="gmail_extra"><div class="gmail_quote">On Sun, Mar 22, 2015 at 7:03 PM, Li, Chen <span dir="ltr"><<a href="mailto:chen.li@intel.com" target="_blank">chen.li@intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal">Hi Sahara,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Recently, I learned Sahara support nodes scale up & scale down, which means “scale” has been limited to node number.<u></u><u></u></p>
<p class="MsoNormal">Is it possible to do a “instance scale”?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">For example, I build a small cluster at first, several slave nodes (running datanode & nodemanager) and a single master node (running all other processes)<u></u><u></u></p>
<p class="MsoNormal">I keep increasing the number of slave nodes, and in a moment, my master node has become the performance bottleneck of the whole cluster.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">In this case, I would like to do several things, such as:<u></u><u></u></p>
<p><u></u><span>1.<span style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:7pt;line-height:normal;font-family:'Times New Roman'">
</span></span><u></u>resize the master node. Using command like “nova resize”, to add more cpu & memory and other resources for this single instance.<u></u><u></u></p>
<p><u></u><span>2.<span style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:7pt;line-height:normal;font-family:'Times New Roman'">
</span></span><u></u>Split processes on the master node to several nodes.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I think it make sense to users.<u></u><u></u></p>
<p class="MsoNormal">Or the whole “performance bottleneck” staff would never happen in real world ???<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Another example, I already have a big cluster, and now I want to enable a new service on it.
<u></u><u></u></p>
<p><u></u><span>1.<span style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:7pt;line-height:normal;font-family:'Times New Roman'">
</span></span><u></u>I would like to start a new node for the new service and add the new node into my cluster.<u></u><u></u></p>
<p><u></u><span>2.<span style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:7pt;line-height:normal;font-family:'Times New Roman'">
</span></span><u></u>Or, just start the new service on a node which already in the cluster.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Currently, Sahara start cluster and manage all nodes based on “templates”, so everything on “living” has to be pre-defined.<u></u><u></u></p>
<p class="MsoNormal">Things on above breaks the whole “pre-defined” thing.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">So, my question here is:<u></u><u></u></p>
<p class="MsoNormal"> Is it possible for Sahara to do things like that ?<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> Would Sahara want to support things like this ?<u></u><u></u></p>
<p class="MsoNormal"> If yes, any plans in the past, in the future?<u></u><u></u></p>
<p class="MsoNormal"> If not, any special reasons ??<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Looking forward to your reply.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thanks.<span class=""><font color="#888888"><u></u><u></u></font></span></p><span class=""><font color="#888888">
<p class="MsoNormal">-chen<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</font></span></div>
</div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div>