<div dir="ltr">Hi,<div><br></div><div>With slight modifications of (2) one can benefit of availability:</div><div>1. There should not be a master node. Each heat engine should be able to act as a master if someone asks it to deploy a template. Current master engine will be responsible to contact other engines and pass them the same template and wait for confirmation.</div>
<div>2. Each Heat engine instance receives whole template but deploys only resources which are designated to engine's zone</div><div><br></div><div>This will provide all benefits of availability as all engines will keep a copy of a template and you can update template by using any heat engine. For example if heat engine in region 1 is down or whole site 1 is down you can update a template with new region settings and update it with using heat engine in region 2.</div>
<div><br></div><div>The multi-region support proposal contains information and proposal for "what" and "how" but does not describe "why". If we talk just about multi-region placement support then option (4) works fine. If there is an intension to build a solution for HA, DR, elastic scalability for spike loads then we need to keep availability as a part of a design.</div>
<div><br></div><div>Thanks</div><div>Georgy</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Nov 15, 2013 at 1:17 PM, Keith Bray <span dir="ltr"><<a href="mailto:keith.bray@rackspace.com" target="_blank">keith.bray@rackspace.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The way I view 2 vs. 4 is that 2 is more complicated and you don't gain<br>
any benefit of availability. If, in 2, your global heat endpoint is down,<br>
you can't update the whole stack. You have to work around it by talking<br>
to Heat (or the individual service endpoints) in the region that is still<br>
alive.<br>
<br>
4 is much simpler in that only one Heat instance is involved. If Heat is<br>
down, you still have just as bad/good workaround, which is you talk to<br>
service endpoints in the region that is still available. If you want to<br>
use Heat in that region to do it, you can Adopt the resources into a Heat<br>
stack in that region. I don't see how 2 is "Robust against failure of<br>
whole region" because if the region on the left part of the picture in 2<br>
goes down, you can't manage your global stack or any of the resources in<br>
the left region that are part of that global stack. All you could manage<br>
is a subset of resources by manipulating the substack in the right region,<br>
but you can do this in 4 as well using Adopt. 4 is a simpler starting use<br>
case and easier (IMO) for a user of the system to digest, and has the HUGE<br>
advantage of being able to orchestrate deploying resources to multiple<br>
regions without a service operator having to have Heat setup and installed<br>
in EVERY region. This is particular important for a private cloud/heat<br>
person trying to deploy to public cloud.. If doing so requires the public<br>
cloud operator have Heat running, then you can't deploy there. If no Heat<br>
in that region is required, then you can use your own instance of Heat to<br>
deploy to any available openstack cloud. That is a HUGE benefit of 4.<br>
<span class="HOEnZb"><font color="#888888"><br>
-Keith<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On 11/15/13 2:58 PM, "Zane Bitter" <<a href="mailto:zbitter@redhat.com">zbitter@redhat.com</a>> wrote:<br>
<br>
>On 15/11/13 18:24, Bartosz Górski wrote:<br>
>> Hi Thomas,<br>
>><br>
>> Each of the two engines will be able to create resources in both<br>
>>regions.<br>
>> We do not need to add anything in the heat client.<br>
>><br>
>> Right now when you want to create a new stack (using heat client or<br>
>> directly API) you need to provide:<br>
>> - stack name<br>
>> - template<br>
>> - parameters (if need)<br>
>> - tenant<br>
>> - username<br>
>> - password<br>
>> - region name (optional)<br>
>><br>
>> The last four (tenant, username, password and region_name) we will call<br>
>> default context.<br>
>> This context is used in Heat to configure all the openstack clients to<br>
>> other service.<br>
>> Username, password and tenant is used for authentication.<br>
>> Region name is use to get appropriate API endpoint from keystone catalog<br>
>> for other openstack service (like nova).<br>
>> In case with one region you do not need to specify it because there is<br>
>> only one endpoint for each service.<br>
>> In multi-region case we have more than one and region name is used to<br>
>> get correct one.<br>
>><br>
>> Each nested stack have its own set of openstack clients (nova client,<br>
>> neutron client, ... etc.) inside the heat engine.<br>
>> Right now for all of them the default context is used to configure<br>
>> clients which will be used to create resources.<br>
>> There is not option to change the default context for now. What I'm<br>
>> trying to do it to add possibility to define different<br>
>> context inside the template file. New context can be passed to nested<br>
>> stack resource to create clients set with different<br>
>> endpoints to call. Heat engine will get appropriate endpoints from<br>
>> keystone catalog for specified region name.<br>
>><br>
>> So from heat engine point of view there is not big change in the<br>
>> workflow. Heat will parse the template, create the<br>
>> dependencies graph and start creating resources in the same way as<br>
>> usual. When he will need to created nested<br>
>> stack with different context he will just use different set of openstack<br>
>> clients (ex. will call services in other region).<br>
>><br>
>> So to sum up each of the two heat engine will be able to create<br>
>> resources in both regions if different context will<br>
>> be specified. If only default context will be used heat will create all<br>
>> resource in the same region where it is located.<br>
><br>
>So, to be clear, this is option (4) from the diagram I put together here:<br>
><a href="https://wiki.openstack.org/wiki/Heat/Blueprints/Multi_Region_Support_for_H" target="_blank">https://wiki.openstack.org/wiki/Heat/Blueprints/Multi_Region_Support_for_H</a><br>
>eat/The_Missing_Diagram<br>
><br>
>It's got a couple of major problems:<br>
><br>
>* When a whole region goes down, you can lose access to the Heat<br>
>instance that was managing still-available resources. This makes it more<br>
>or less impossible to use Heat to manage a highly-available global<br>
>application.<br>
><br>
>* Instances have to communicate back to the Heat instance that owns them<br>
>(e.g. for WaitConditions), and it's not yet clear that this is feasible<br>
>in general.<br>
><br>
>There are also a number of other things I really don't like about this<br>
>solution (listed on the wiki page), though reasonable people may disagree.<br>
><br>
>cheers,<br>
>Zane.<br>
><br>
>_______________________________________________<br>
>OpenStack-dev mailing list<br>
><a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Georgy Okrokvertskhov<br>
Technical Program Manager,<br>Cloud and Infrastructure Services,<br>
Mirantis<br>
<a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com</a><br>
Tel. +1 650 963 9828<br>
Mob. +1 650 996 3284<br>
</div>