<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Sep 26, 2014 at 2:01 PM, Angus Lees <span dir="ltr"><<a href="mailto:gus@inodes.org" target="_blank">gus@inodes.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="">On Thu, 25 Sep 2014 04:01:38 PM Fox, Kevin M wrote:<br>
> Doesn't nova with a docker driver and heat autoscaling handle case 2 and 3<br>
> for control jobs? Has anyone tried yet?<br>
<br>
</span>For reference, the cases were:<br>
<span class=""><br>
> - Something to deploy the code (docker / distro packages / pip install /<br>
> etc)<br>
> - Something to choose where to deploy<br>
> - Something to respond to machine outages / autoscaling and re-deploy as<br>
> necessary<br>
<br>
<br>
</span>I tried for a while, yes. The problems I ran into (and I'd be interested to<br>
know if there are solutions to these):<br>
<br>
- I'm trying to deploy into VMs on rackspace public cloud (just because that's<br>
what I have). This means I can't use the nova docker driver, without<br>
constructing an entire self-contained openstack undercloud first.<br>
<br>
- heat+cloud-init (afaics) can't deal with circular dependencies (like nova<-<br>
>neutron) since the machines need to exist first before you can refer to their<br>
IPs.<br>
>From what I can see, TripleO gets around this by always scheduling them on the<br>
same machine and just using the known local IP. Other installs declare fixed<br>
IPs up front - on rackspace I can't do that (easily).<br>
I can't use loadbalancers via heat for this because the loadbalancers need to<br>
know the backend node addresses, which means the nodes have to exist first and<br>
you're back to a circular dependency.<br>
<br>
For comparision, with kubernetes you declare the loadbalancer-equivalents<br>
(services) up front with a search expression for the backends. In a second<br>
pass you create the backends (pods) which can refer to any of the loadbalanced<br>
endpoints. The loadbalancers then reconfigure themselves on the fly to find the<br>
new backends. You _can_ do a similar lazy-loadbalancer-reconfig thing with<br>
openstack too, but not with heat and not just "out of the box".<br></blockquote><div><br></div><div>Do you have a minimal template that shows what you are trying to do?<br></div><div>(just to demonstrate the circular dependency). <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
- My experiences using heat for anything complex have been extremely<br>
frustrating. The version on rackspace public cloud is ancient and limited,<br>
and quite easy to get into a state where the only fix is to destroy the entire<br>
stack and recreate it. I'm sure these are fixed in newer versions of heat, but<br>
last time I tried I was unable to run it standalone against an arms-length<br>
keystone because some of the recursive heat callbacks became confused about<br>
which auth token to use.<br></blockquote><div><br></div><div>Gus we are working at improving standalone (Steven Baker has some patch out for this).<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
(I'm sure this can be fixed, if it wasn't already just me using it wrong in the<br>
first place.)<br>
<br>
- As far as I know, nothing in a heat/loadbalancer/nova stack will actually<br>
reschedule jobs away from a failed machine. There's also no lazy<br></blockquote><div><br>This might go part of the way there, the other part of it is detecting the failed machine<br>and some how marking it as failed.<br> <a href="https://review.openstack.org/#/c/105907/">https://review.openstack.org/#/c/105907/</a><br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
discovery/nameservice mechanism, so updating IP address declarations in cloud-<br>
configs tend to ripple through the heat config and cause all sorts of<br>
VMs/containers to be reinstalled without any sort of throttling or rolling<br>
update.<br>
<br>
<br>
So: I think there's some things to learn from the kubernetes approach, which<br>
is why I'm trying to gain more experience with it. I know I'm learning more<br>
about the various OpenStack components along the way too ;)<br></blockquote><div><br></div><div>This is valuable feedback, we need to improve Heat to make these use case work better.<br></div><div>But I also don't believe there is one tool for all jobs, so see little harm in trying<br></div><div>other things out too.<br></div><div><br></div><div>Thanks<br></div><div>Angus<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class=""><div class="h5"><br>
--<br>
- Gus<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div></div>