Open Stack

Hello Stan,
Please see comments inline.
Cheers,
Patrick
On 10/23/13 8:33 PM, Stan Lagun wrote:
> Hi Patric,
>
> Thank you for such great post! This is very close to the vision I've 
> tried to propose earlier on software orchestration thread and I'm glad 
> other people concern about the same issues. However the problem the 
> problem with PaaS-like approached it that they currently on a little 
> bit higher abstraction layer than Heat is intended to be. Typical Heat 
> users are more of DevOps people rather than those who enjoy 
> PaaS-related solutions. Going that direction would require some major 
> paradigm shift for the Heat which I think is unnecessary.
Okay. But don't get me wrong. I am not militating for embarking 
PaaS-like capabilities into Heat. Far from it. There are two basic 
reasons for that. There are to many ways of approaching the PaaS 
endeavor and that would kill innovation for those who are trying to 
build value atop of OpenStack/Heat like ourselves. Even though we are 
DevOps the intent is that our users don't have to be since we provide 
them with built-in middleware stacks covering some verticals 
(high-performance computing related) that power users can leverage 
out-of-the-box to deploy their own apps. So, I guess what I intended to 
say is; let's try to keep it lean. Do not over engineer this thing with 
nuts and bolts allover the place because Heat is and will be 
increasingly used in completely unexpected ways.
>
> I believe there is a place in OpenStack software-orchestration 
> ecosystem for layers that would sin on top of Heat and provide more 
> high-level services for software composition, dependency management. 
> Heat is not aimed to be software-everything. I would suggest you to 
> take a look at Murano project as it is very very close to what you 
> want to achieve and as every open-source project it needs community 
> contributions. And I believe that it is the place in OpenStack 
> ecosystem where your expirience would be most valuable and appreciated 
> as well as your contributions
Thank you for the invitation! We also welcome you to work with us on the 
XLcloud project which is also open-source Apache V2 project. Java-based 
though. Nobody is perfect ;-). More seriously we are thinking of moving 
the code to github and apply for incubation eventually making the 
OpenStack community become bigger and richer by joining in with the Java 
community :-)

The code
http://gitorious.ow2.org/xlcloud
A beginning of user documentation can be found here:
https://129.184.11.121:8443/display/XGM/XLcloud+Guides+and+Manuals+Home

>
>
> On Wed, Oct 23, 2013 at 9:58 PM, Patrick Petit <patrick.petit at bull.net 
> <mailto:patrick.petit at bull.net>> wrote:
>
>     Dear Steve and All,
>
>     If I may add up on this already busy thread to share our
>     experience with using Heat in large and complex software deployments.
>
>     I work on a project which precisely provides additional value at
>     the articulation point between resource orchestration automation
>     and configuration management. We rely on Heat and chef-solo
>     respectively for these base management functions. On top of this,
>     we have developed an event-driven workflow to manage the
>     life-cycles of complex software stacks which primary purpose is to
>     support middleware components as opposed to end-user apps. Our use
>     cases are peculiar in the sense that software setup (install,
>     config, contextualization) is not a one-time operation issue but a
>     continuous thing that can happen any time in life-span of a stack.
>     Users can deploy (and undeploy) apps long time after the stack is
>     created. Auto-scaling may also result in an asynchronous apps
>     deployment. More about this latter. The framework we have designed
>     works well for us. It clearly refers to a PaaS-like environment
>     which I understand is not the topic of the HOT software
>     configuration proposal(s) and that's absolutely fine with us.
>     However, the question for us is whether the separation of software
>     config from resources would make our life easier or not. I think
>     the answer is definitely yes but at the condition that the DSL
>     extension preserves almost everything from the expressiveness of
>     the resource element. In practice, I think that a strict
>     separation between resource and component will be hard to achieve
>     because we'll always need a little bit of application's specific
>     in the resources. Take for example the case of the SecurityGroups.
>     The ports open in a SecurityGroup are application specific.
>
>     Then, designing a Chef or Puppet component type may be harder than
>     it looks at first glance. Speaking of our use cases we still need
>     a little bit of scripting in the instance's user-data block to
>     setup a working chef-solo environment. For example, we run
>     librarian-chef prior to starting chef-solo to resolve the cookbook
>     dependencies. A cookbook can present itself as a downloadable
>     tarball but it's not always the case. A chef component type would
>     have to support getting a cookbook from a public or private git
>     repo (maybe subversion), handle situations where there is one
>     cookbook per repo or multiple cookbooks per repo, let the user
>     choose a particular branch or label, provide ssh keys if it's a
>     private repo, and so forth. We support all of this scenarios and
>     so we can provide more detailed requirements if needed.
>
>     I am not sure adding component relations like the 'depends-on'
>     would really help us since it is the job of config management to
>     handle software dependencies. Also, it doesn't address the issue
>     of circular dependencies. Circular dependencies occur in complex
>     software stack deployments. Example. When we setup a Slum virtual
>     cluster, both the head node and compute nodes depend on one
>     another to complete their configuration and so they would wait for
>     each other indefinitely if we were to rely on the 'depends-on'. In
>     addition, I think it's critical to distinguish between
>     configuration parameters which are known ahead of time, like a db
>     name or user name and password, versus contextualization
>     parameters which are known after the fact generally when the
>     instance is created. Typically those contextualization parameters
>     are IP addresses but not only. The fact packages x,y,z have been
>     properly installed and services a,b,c successfully started is
>     contextualization information (a.k.a facts) which may be
>     indicative that other components can move on to the next setup stage.
>
>     The case of complex deployments with or without circular
>     dependencies is typically resolved by making the system converge
>     toward the desirable end-state through running idempotent recipes.
>     This is our approach. The first configuration phase handles
>     parametrization which in general brings an instance to
>     CREATE_COMPLETE state. A second phase follows to handle
>     contextualization at the stack level. As a matter of fact, a new
>     contextualization should be triggered every time an instance
>     enters or leave the CREATE_COMPLETE state which may happen any
>     time with auto-scaling. In that phase, circular dependencies can
>     be resolved because all contextualization data can be compiled
>     globally. Notice that Heat doesn't provide a purpose built
>     resource or service like Chef's data-bag for the storage and
>     retrieval of metadata. This a gap which IMO should be addressed in
>     the proposal. Currently, we use a kludge that is to create a fake
>     AWS::AutoScaling::LaunchConfiguration resource to store
>     contextualization data in the metadata section of that resource.
>
>     Aside from the HOT software configuration proposal(s). There are
>     two critical enhancements in Heat that would make software
>     life-cycles management much easier. In fact, they are actual
>     blockers for us.
>
>     The first one would be to support asynchronous notifications when
>     an instance is created or deleted as a result of an auto-scaling
>     decision. As stated earlier, contextualization needs to apply in a
>     stack every time a instance enters or leaves the CREATE_COMPLETE
>     state. I am not referring to a Ceilometer notification but a Heat
>     notification that can be consumed by a Heat client.
>
>     The second one would be to support a new type of AWS::IAM::User
>     (perhaps OS::IAM::User) resource whereby one could pass Keystone
>     credentials to be able to specify Ceilometer alarms based on
>     application's specific metrics (a.k.a KPIs).
>
>     I hope this is making sense to you and can serve as a basis for
>     further discussions and refinements.
>
>     Cheers,
>     Patrick
>
>
>     On 10/16/13 12:48 AM, Steve Baker wrote:
>>     I've just written some proposals to address Heat's HOT software
>>     configuration needs, and I'd like to use this thread to get some
>>     feedback:
>>     https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
>>     https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config
>>
>>     Please read the proposals and reply to the list with any comments
>>     or suggestions.
>>
>>     We can spend some time discussing software configuration at
>>     tomorrow's Heat meeting, but I fully expect we'll still be in the
>>     discussion phase at Hong Kong.
>>
>>     cheers
>>
>>
>>     _______________________________________________
>>     OpenStack-dev mailing list
>>     OpenStack-dev at lists.openstack.org  <mailto:OpenStack-dev at lists.openstack.org>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>     -- 
>     Patrick Petit
>     Cloud Computing Principal Architect, Innovative Products
>     Bull, Architect of an Open World TM
>     Tél : +33 (0)4 76 29 70 31
>     Mobile : +33 (0)6 85 22 06 39
>     http://www.bull.com
>
>
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> -- 
> Sincerely yours
> Stanislav (Stan) Lagun
> Senior Developer
> Mirantis
> 35b/3, Vorontsovskaya St.
> Moscow, Russia
> Skype: stanlagun
> www.mirantis.com <http://www.mirantis.com/>
> slagun at mirantis.com <mailto:slagun at mirantis.com>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
Patrick Petit
Cloud Computing Principal Architect, Innovative Products
Bull, Architect of an Open World TM
Tél : +33 (0)4 76 29 70 31
Mobile : +33 (0)6 85 22 06 39
http://www.bull.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131024/2b041bfa/attachment.html>