Open Stack

On Tue, Jul 30, 2013 at 3:56 AM, Ruslan Kamaldinov <rkamaldinov at mirantis.com
> wrote:

> This question was asked several times. So we decided to provide a
> detailed response.
>
> 1. The first question is “Why doesn’t Savanna use Heat to provision VMs?”
>
> Generally using Heat underneath for infrastructure provisioning looks
> reasonable. In a tactic perspective there are few factors making Heat
> usage underneath Savanna problematic:
> * Heat stability for Grizzly release. Savanna currently maintains
> Grizzly+ compatibility.
>

This shouldn't be an issue anymore. Now that we are in Havana


> * Installation of large Hadoop clusters (100+ nodes). Will be
> addressed by proposed architecture changes.
>

What about heat doesn't work for this?


> * Anti-affinity support for HDFS redundancy in cloud environment
>

Why can't heat do this


> * Circular dependencies - we should generate ‘/etc/hosts’ for all
> instances in provisioned cluster. We can’t use cloud init for this
> directly. There are a couple possible solutions using Heat, but none
> of them looks like a straightforward solution.
>

So why not make heat do this better


> * Level of complexity. We try to keep things as simple as possible.
> Adding extra layer will increase overall complexity of the solution.
> In addition both Savanna and Heat under active development changing
> lots of internals and even APIs and will require extra effort to
> coordinate.
>

But coordinating will result in less duplicate work


>
> Here is what we’ll do:
> * Create a wiki page with text from this email
> * Create a list of requirements for Heat
>
> Once Heat fulfills all the requirements we will be able and should use
> Heat for VM provisioning.
>
>
Now that you have a basic working version of Savana, I would prefer to see
you help make heat fit work for you instead of reimplementing a subset of
what it is trying to do.



>
> 2. Let’s answer the second question - why we need Savanna? Can’t we
> use Heat to do what Savanna does?
>
> * Savanna provides bunch of Hadoop-specific features. It’ll be hard to
> provide them as Heat plugin
> * Savanna provides Hadoop-specific APIs and functionality. Heat use
> cases are mostly around provisioning/deployment.
> * Savanna provides integration with various Hadoop distributions
> through pluggable mechanism
>
> Now, more details on each item.
> Hadoop specific features:
> * Tight Swift integration. Hadoop can read and write from/to Swift
> object storage. Savanna provides required configs for the Hadoop
> cluster.
> * Usage of anti-affinity to preserve data-redundancy of HDFS nodes
>
> Hadoop-specific APIs and functionality:
> * Hadoop cluster scaling
> * Elastic Data Processing: https://wiki.openstack.org/wiki/Savanna/EDP
>
> Integration with Hadoop distributions through pluggable mechanism:
> - Usually Hadoop cluster deployment is a multi-step operation. First
> step is to install management console (for instance Apache Ambari).
> Second step is to communicate with management console through REST API
> to provision Hadoop on the cluster. Savanna wraps all this operations
> under well-defined API.
>
> I hope all the items above explain why we need Savanna as a separate
> OpenStack service.
>
>
> 3. Why can’t Savanna be used as a plugin for Heat?
> It should be and it will be someday.
>
>
> Regards,
> Ruslan
>
> On Thu, Jul 25, 2013 at 7:42 PM, Joe Gordon <joe.gordon0 at gmail.com> wrote:
> >
> > On Jul 23, 2013 12:34 PM, "Sergey Lukjanov" <slukjanov at mirantis.com>
> wrote:
> >>
> >> Hi evereyone,
> >>
> >> We’ve started working on upgrading Savanna architecture in version 0.3
> to
> >> make it horizontally scalable.
> >>
> >> The most part of information is in the wiki page -
> >> https://wiki.openstack.org/wiki/Savanna/NextGenArchitecture.
> >>
> >> Additionally there are several blueprints created for this activity -
> >> https://blueprints.launchpad.net/savanna?searchtext=ng-
> >>
> >> We are looking for comments / questions / suggestions.
> >
> > This sounds like most of this can be built around Heat, except maybe the
> > rest api to hadoop.  So why not use heat for the deploy part?
> >
> >>
> >> P.S. The another thing that we’re working on in Savanna 0.3 is EDP
> >> (Elastic Data Processing).
> >>
> >> Thank you!
> >>
> >> Sincerely yours,
> >> Sergey Lukjanov
> >> Savanna Technical Lead
> >> Mirantis Inc.
> >>
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130730/f6a2f20f/attachment.html>