[openstack-dev] [Nova] bp proposal: discovery of peer instances through metadata service

Justin Santa Barbara justin at fathomdb.com
Fri Jan 24 20:29:49 UTC 2014

Clint Byrum <clint at fewbar.com> wrote:

> Heat has been working hard to be able to do per-instance limited access
> in Keystone for a while. A trust might work just fine for what you want.

I wasn't actually aware of the progress on trusts.  It would be helpful
except (1) it is more work to have to create a separate trust (it is even
more painful to do so with IAM) and (2) it doesn't look like we can yet
lock-down these delegations as much as people would probably want.  I think
IAM is the end-game in terms of the model that people actually want, and it
ends up being incredibly complex.  Delegation is very useful (particularly
because clusters could auto-scale themselves), but I'd love to get an
easier solution for the peer discovery problem than where delegation ends

Are you hesitant to just use Heat? This is exactly what it is supposed
> to do.. make a bunch of API calls and expose the results to instances
> for use in configuration.

> If you're just hesitant to use a declarative templating language, I
> totally understand. The auto-scaling minded people are also feeling
> this way. You could join them in the quest to create an imperative
> cluster-making API for Heat.

I don't want to _depend_ on Heat.  My hope is that we can just launch 3
instances with the Cassandra image, and get a Cassandra cluster.  It might
be that we want Heat to auto-scale that cluster, Ceilometer to figure out
when to scale it, Neutron to isolate it, etc but I think we can solve the
basic discovery problem cleanly without tying in all the other services.
 Heat's value-add doesn't come from solving this problem!

:) We are on the same page. I really think Heat is where higher level
> information sharing of this type belongs. I do think it might make sense
> for Heat to push things into user-data post-boot, rather than only expose
> them via its own metadata service. However, even without that, you can
> achieve what you're talking about right now with Heat's separate metadata.

> N instances in one API call is something Heat does well, and it does
> auto scaling too, so I feel like your idea is mostly just asking for a
> simpler way to use Heat, which I think everyone would agree would be
> good for all Heat users. :)

I have a personal design goal of solving the discovery problem in a way
that works even on non-clouds.  So I can write a clustered service, and it
will run everywhere.  The way I see it is that:

   - If we're on physical, the instance will use multicast & broadcast to
   find peers on the network.
   - If we're on OpenStack, the instance will use this blueprint to find
   its peers.  The instance may be launched through Nova, or Heat, or
   Puppet/Chef/Salt/etc.  I would like to see people use Heat, but I don't
   want to force people to use Heat.  If Heat starts putting a more accurate
   list of peers into metadata, I will check that first.  But if I can't find
   that list of peers that Heat provides, I will fall-back to whatever I can
   get from Nova so that I can cope with people not on Heat.
   - If we're on EC2, the user must configure an IAM role and assign it to
   their instances, and then we will query the list of instances.

It gives me great pleasure that EC2 will end up needing the most
undifferentiated lifting from the user.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140124/4e06b7a9/attachment.html>

More information about the OpenStack-dev mailing list