<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 11/26/2013 06:21 AM, Sergey Lukjanov
wrote:<br>
</div>
<blockquote
cite="mid:F91C564D-964B-407E-BA16-B12E16FC4FE8@mirantis.com"
type="cite">
<pre wrap="">Hi guys,
There was the Design Summit session in Hong Kong about Heat integration and Savanna scalability [0]. We discussed some details about it, approved integration plan and decided to use guest agents.
First of all, or the Icehouse release cycle, we’ll implement resources orchestration using Heat by creating YAML templates generator, blueprints [1][2] and PoC [3]. It’ll be done by implementing extension mechanism for provisioning w/o removing current orchestration solution to transparently replace current code with the new Heat-based approach. As the first step all resources (VMs, volumes, IPs) will be provisioned by Heat using template generated by Savanna. Hadoop configuration will be done by Savanna and especially by corresponding plugins.
The second step of improving provisioning code will be to implement guest agent for Savanna (we’re looking at unified agent [4][5] implementation due to the growing amount of projects interested in it). Guest agents will allow Savanna plugins to configure software by interacting with vendor-specific management console APIs. The main goal of implementing agents in Savanna is to get rid of direct ssh and http access to VMs.
For the earlier J release cycle we’re planning to enable Heat by default and then completely remove our current direct provisioning. We’ll contribute Savanna resource to Heat, it’ll be something like “Data Processing Cluster” or just “Hadoop Cluster” at the beginning, I’ll start discussion on it separately.
There are some problems that we’ll try to solve to support all current Savanna features:
* anti-affinity support (currently implemented using scheduler hints ‘not on the specific hosts’ and stack provisioning is simultaneous in this case); there are two possible solutions - use Nova’s Group API (when it’ll be ready) or add support for it into the Heat; </pre>
</blockquote>
OS::Nova::Server has the scheduler_hints property, so you could
always continue with the current approach in the interim.<br>
<blockquote
cite="mid:F91C564D-964B-407E-BA16-B12E16FC4FE8@mirantis.com"
type="cite">
<pre wrap="">
* partially active stack and/or optional and mandatory resources; the easiest way to explain this is to have an example - we provisioning 100 nodes with same roles (data nodes of Hadoop cluster) and only one is down, so, we can say that cluster is partially active and then rebuild failed nodes.
</pre>
</blockquote>
Some combination of our new autoscaling and stack convergence should
help here.<br>
<blockquote
cite="mid:F91C564D-964B-407E-BA16-B12E16FC4FE8@mirantis.com"
type="cite">
<pre wrap="">
To summarize, we’re currently finishing the PoC version of Heat-based provisioning and we’ll merge it into the codebase soon.
[0] <a class="moz-txt-link-freetext" href="https://etherpad.openstack.org/p/savanna-icehouse-architecture">https://etherpad.openstack.org/p/savanna-icehouse-architecture</a>
[1] <a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/savanna/+spec/heat-backed-resources-provisioning">https://blueprints.launchpad.net/savanna/+spec/heat-backed-resources-provisioning</a>
[2] <a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/savanna/+spec/infra-provisioning-extensions">https://blueprints.launchpad.net/savanna/+spec/infra-provisioning-extensions</a>
[3] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/55978">https://review.openstack.org/#/c/55978</a>
[4] <a class="moz-txt-link-freetext" href="http://lists.openstack.org/pipermail/openstack-dev/2013-November/018276.html">http://lists.openstack.org/pipermail/openstack-dev/2013-November/018276.html</a>
[5] <a class="moz-txt-link-freetext" href="https://etherpad.openstack.org/p/UnifiedAgents">https://etherpad.openstack.org/p/UnifiedAgents</a>
</pre>
</blockquote>
Nice, I've just added some comments to
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
<a href="https://review.openstack.org/#/c/55978/">https://review.openstack.org/#/c/55978/</a>
. Feel free to add me as a reviewer to any others.<br>
</body>
</html>