<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hello Stan,<br>
Please see comments inline.<br>
Cheers,<br>
Patrick<br>
On 10/23/13 8:33 PM, Stan Lagun wrote:<br>
</div>
<blockquote
cite="mid:CAOCoZiazFXrw23Bz9FZ7-fVKhXOxupB+QzmVnmJKpc+HoZgKZA@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<div dir="ltr">
<div>
<div>Hi <span name="Patrick Petit" class="">Patric,<br>
<br>
</span></div>
<span name="Patrick Petit" class="">Thank you for such great
post! This is very close to the vision I've tried to propose
earlier on software orchestration thread and I'm glad other
people concern about the same issues. However the problem
the problem with PaaS-like approached it that they currently
on a little bit higher abstraction layer than Heat is
intended to be. Typical Heat users are more of DevOps people
rather than those who enjoy PaaS-related solutions. Going
that direction would require some major paradigm shift for
the Heat which I think is unnecessary. <br>
</span></div>
</div>
</blockquote>
Okay. But don't get me wrong. I am not militating for embarking
PaaS-like capabilities into Heat. Far from it. There are two basic
reasons for that. There are to many ways of approaching the PaaS
endeavor and that would kill innovation for those who are trying to
build value atop of OpenStack/Heat like ourselves. Even though we
are DevOps the intent is that our users don't have to be since we
provide them with built-in middleware stacks covering some verticals
(high-performance computing related) that power users can leverage
out-of-the-box to deploy their own apps. So, I guess what I intended
to say is; let's try to keep it lean. Do not over engineer this
thing with nuts and bolts allover the place because Heat is and will
be increasingly used in completely unexpected ways.<br>
<blockquote
cite="mid:CAOCoZiazFXrw23Bz9FZ7-fVKhXOxupB+QzmVnmJKpc+HoZgKZA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div><span name="Patrick Petit" class="">
</span></div>
<span name="Patrick Petit" class=""><br>
I believe there is a place in OpenStack software-orchestration
ecosystem for layers that would sin on top of Heat and provide
more high-level services for software composition, dependency
management. Heat is not aimed to be software-everything. I
would suggest you to take a look at Murano project as it is
very very close to what you want to achieve and as every
open-source project it needs community contributions. And I
believe that it is the place in OpenStack ecosystem where your
expirience would be most valuable and appreciated as well as
your contributions <br>
</span></div>
</blockquote>
Thank you for the invitation! We also welcome you to work with us on
the XLcloud project which is also open-source Apache V2 project.
Java-based though. Nobody is perfect ;-). More seriously we are
thinking of moving the code to github and apply for incubation
eventually making the OpenStack community become bigger and richer
by joining in with the Java community :-)<br>
<br>
The code<br>
<a class="moz-txt-link-freetext" href="http://gitorious.ow2.org/xlcloud">http://gitorious.ow2.org/xlcloud</a><br>
A beginning of user documentation can be found here:<br>
<a class="moz-txt-link-freetext" href="https://129.184.11.121:8443/display/XGM/XLcloud+Guides+and+Manuals+Home">https://129.184.11.121:8443/display/XGM/XLcloud+Guides+and+Manuals+Home</a><br>
<br>
<blockquote
cite="mid:CAOCoZiazFXrw23Bz9FZ7-fVKhXOxupB+QzmVnmJKpc+HoZgKZA@mail.gmail.com"
type="cite">
<div dir="ltr"><span name="Patrick Petit" class=""> </span></div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Oct 23, 2013 at 9:58 PM,
Patrick Petit <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:patrick.petit@bull.net" target="_blank">patrick.petit@bull.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Dear Steve and All,<br>
<br>
If I may add up on this already busy thread to share our
experience with using Heat in large and complex software
deployments.<br>
<br>
I work on a project which precisely provides additional
value at the articulation point between resource
orchestration automation and configuration management.
We rely on Heat and chef-solo respectively for these
base management functions. On top of this, we have
developed an event-driven workflow to manage the
life-cycles of complex software stacks which primary
purpose is to support middleware components as opposed
to end-user apps. Our use cases are peculiar in the
sense that software setup (install, config,
contextualization) is not a one-time operation issue but
a continuous thing that can happen any time in life-span
of a stack. Users can deploy (and undeploy) apps long
time after the stack is created. Auto-scaling may also
result in an asynchronous apps deployment. More about
this latter. The framework we have designed works well
for us. It clearly refers to a PaaS-like environment
which I understand is not the topic of the HOT software
configuration proposal(s) and that's absolutely fine
with us. However, the question for us is whether the
separation of software config from resources would make
our life easier or not. I think the answer is definitely
yes but at the condition that the DSL extension
preserves almost everything from the expressiveness of
the resource element. In practice, I think that a strict
separation between resource and component will be hard
to achieve because we'll always need a little bit of
application's specific in the resources. Take for
example the case of the SecurityGroups. The ports open
in a SecurityGroup are application specific.<br>
<br>
Then, designing a Chef or Puppet component type may be
harder than it looks at first glance. Speaking of our
use cases we still need a little bit of scripting in the
instance's user-data block to setup a working chef-solo
environment. For example, we run librarian-chef prior to
starting chef-solo to resolve the cookbook dependencies.
A cookbook can present itself as a downloadable tarball
but it's not always the case. A chef component type
would have to support getting a cookbook from a public
or private git repo (maybe subversion), handle
situations where there is one cookbook per repo or
multiple cookbooks per repo, let the user choose a
particular branch or label, provide ssh keys if it's a
private repo, and so forth. We support all of this
scenarios and so we can provide more detailed
requirements if needed.<br>
<br>
I am not sure adding component relations like the
'depends-on' would really help us since it is the job of
config management to handle software dependencies. Also,
it doesn't address the issue of circular dependencies.
Circular dependencies occur in complex software stack
deployments. Example. When we setup a Slum virtual
cluster, both the head node and compute nodes depend on
one another to complete their configuration and so they
would wait for each other indefinitely if we were to
rely on the 'depends-on'. In addition, I think it's
critical to distinguish between configuration parameters
which are known ahead of time, like a db name or user
name and password, versus contextualization parameters
which are known after the fact generally when the
instance is created. Typically those contextualization
parameters are IP addresses but not only. The fact
packages x,y,z have been properly installed and services
a,b,c successfully started is contextualization
information (a.k.a facts) which may be indicative that
other components can move on to the next setup stage.<br>
<br>
The case of complex deployments with or without circular
dependencies is typically resolved by making the system
converge toward the desirable end-state through running
idempotent recipes. This is our approach. The first
configuration phase handles parametrization which in
general brings an instance to CREATE_COMPLETE state. A
second phase follows to handle contextualization at the
stack level. As a matter of fact, a new
contextualization should be triggered every time an
instance enters or leave the CREATE_COMPLETE state which
may happen any time with auto-scaling. In that phase,
circular dependencies can be resolved because all
contextualization data can be compiled globally. Notice
that Heat doesn't provide a purpose built resource or
service like Chef's data-bag for the storage and
retrieval of metadata. This a gap which IMO should be
addressed in the proposal. Currently, we use a kludge
that is to create a fake
AWS::AutoScaling::LaunchConfiguration resource to store
contextualization data in the metadata section of that
resource.<br>
<br>
Aside from the HOT software configuration proposal(s).
There are two critical enhancements in Heat that would
make software life-cycles management much easier. In
fact, they are actual blockers for us.<br>
<br>
The first one would be to support asynchronous
notifications when an instance is created or deleted as
a result of an auto-scaling decision. As stated earlier,
contextualization needs to apply in a stack every time a
instance enters or leaves the CREATE_COMPLETE state. I
am not referring to a Ceilometer notification but a Heat
notification that can be consumed by a Heat client.<br>
<br>
The second one would be to support a new type of
AWS::IAM::User (perhaps OS::IAM::User) resource whereby
one could pass Keystone credentials to be able to
specify Ceilometer alarms based on application's
specific metrics (a.k.a KPIs).<br>
<br>
I hope this is making sense to you and can serve as a
basis for further discussions and refinements.<br>
<br>
Cheers,<br>
Patrick
<div>
<div class="h5"><br>
<br>
On 10/16/13 12:48 AM, Steve Baker wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="h5"> I've just written some proposals to
address Heat's HOT software configuration needs, and
I'd like to use this thread to get some feedback:<br>
<a moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config"
target="_blank">https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config</a><br>
<a moz-do-not-send="true"
href="https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config"
target="_blank">https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config</a><br>
<br>
Please read the proposals and reply to the list with
any comments or suggestions.<br>
<br>
We can spend some time discussing software
configuration at tomorrow's Heat meeting, but I
fully expect we'll still be in the discussion phase
at Hong Kong.<br>
<br>
cheers<br>
<br>
<fieldset></fieldset>
<br>
</div>
</div>
<div class="im">
<pre>_______________________________________________
OpenStack-dev mailing list
<a moz-do-not-send="true" href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</div>
</blockquote>
<span class="HOEnZb"><font color="#888888"> <br>
<br>
<pre cols="72">--
Patrick Petit
Cloud Computing Principal Architect, Innovative Products
Bull, Architect of an Open World TM
Tél : +33 (0)4 76 29 70 31
Mobile : +33 (0)6 85 22 06 39
<a moz-do-not-send="true" href="http://www.bull.com" target="_blank">http://www.bull.com</a></pre>
</font></span></div>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div dir="ltr"><span
style="border-collapse:separate;color:rgb(0,0,0);font-family:'Times
New
Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span
style="font-family:arial;font-size:small">Sincerely yours<br>
Stanislav (Stan) Lagun<br>
Senior Developer<br>
Mirantis</span></span><br>
<span
style="border-collapse:separate;color:rgb(0,0,0);font-family:'Times
New
Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span
style="font-family:arial;font-size:small"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif""
lang="EN-US">35b/3, Vorontsovskaya
St.</span><br>
Moscow, Russia<br>
Skype: stanlagun<br>
<a moz-do-not-send="true" href="http://www.mirantis.com/"
target="_blank">www.mirantis.com</a><br>
<a moz-do-not-send="true"
href="mailto:slagun@mirantis.com" target="_blank">slagun@mirantis.com</a></span></span></div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Patrick Petit
Cloud Computing Principal Architect, Innovative Products
Bull, Architect of an Open World TM
Tél : +33 (0)4 76 29 70 31
Mobile : +33 (0)6 85 22 06 39
<a class="moz-txt-link-freetext" href="http://www.bull.com">http://www.bull.com</a></pre>
</body>
</html>