<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><span style="color:rgb(34,34,34)">Well if you're on a Neutron private network then you'd only be DDOS-ing yourself.</span><br></div>
In fact I think Neutron allows broadcast and multicast on private networks, and<br>
as nova-net is going to be deprecated at some point I wonder if this is reducing<br>
to a corner case ?</blockquote><div><br></div>Neutron may well re-enable multicast/broadcast, but I think that (1) multicast/broadcast is the wrong thing to use anyway, and more of an artifact of the way clusters were previously deployed and (2) we should have an option that doesn't require people to install Neutron with multicast enabled. I think that many public clouds, particularly those that want to encourage an XaaS ecosystem, will avoid forcing people to use Neutron's isolated networks.<div>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im"><span style="color:rgb(34,34,34)">it seems that IP address would really be enough</span><br>
</div>
and the agents or whatever in the instance could take it from there ?<br></blockquote><div><br></div><div>Quite possibly. I'm very open to doing just that if people would prefer.</div><div><br class=""><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
What worried me most, I think, is that if we make this part of the standard<br>
metadata then everyone would get it, and that raises a couple of concerns:<br>
<br>
- Users with lots of instances (say 1000's) but who weren't trying to run any form<br>
of discovery would start getting a lot more metadata returned, which might cause<br>
performance issues<br></blockquote><div><br></div><div>The list of peers is only returned if the request comes in for peers.json, so there's no growth in the returned data unless it is requested. Because of the very clear instructions in the comment to always pre-fetch data, it is always pre-fetched, even though it would make more sense to me to fetch it lazily when it was requested! Easy to fix, but I'm obeying the comment because it was phrased in the form of a grammatically valid sentence :-)</div>
<div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
- Some users might be running instances on behalf of customers (consider say a<br>
PaaS type service where the user gets access into an instance but not to the<br>
Nova API. In that case I wouldn't want one instance to be able to discover these<br>
kinds of details about other instances.<br>
<br></blockquote><div><br></div><div>Yes, I do think this is a valid concern. But, there is likely to be _much_ more sensitive information in the metadata service, so anyone doing this is hopefully blocking the metadata service anyway. On EC2 with IAM, or if we use trusts, there will be auth token in there. And not just for security, but also because if the PaaS program is auto-detecting EC2/OpenStack by looking for the metadata service, that will cause the program to be very confused if it sees the metadata for its host!</div>
<div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">So it kind of feels to me that this should be some other specific set of metadata<br>
that instances can ask for, and that instances have to explicitly opt into.<br></blockquote><div><br></div><div>I think we have this in terms of the peers.json endpoint for byte-count concerns. For security, we only go per-project; I don't think we're exposing any new information; and anyone doing multi-tenant should either be using projects or be blocking 169.254 anyway.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">We already have a mechanism now where an instance can push metadata as a<br>
way of Windows instances sharing their passwords - so maybe this could build<br>
on that somehow - for example each instance pushes the data its willing to share<br>
with other instances owned by the same tenant ?<br></blockquote><div><br></div><div>I do like that and think it would be very cool, but it is much more complex to implement I think. It also starts to become a different problem: I do think we need a state-store, like Swift or etcd or Zookeeper that is easily accessibly to the instances. Indeed, one of the things I'd like to build using this blueprint is a distributed key-value store which would offer that functionality. But I think that having peer discovery is a much more tightly defined blueprint, whereas some form of shared read-write data-store is probably top-level project complexity.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br>
> On external agents doing the configuration: yes, they could put this into user<br>
> defined metadata, but then we're tied to a configuration system. We have<br>
> to get 20 configuration systems to agree on a common format (Heat, Puppet,<br>
> Chef, Ansible, SaltStack, Vagrant, Fabric, all the home-grown systems!) It<br>
> also makes it hard to launch instances concurrently (because you want node<br>
> #2 to have the metadata for node #1, so you have to wait for node #1 to get<br>
> an IP).<br>
><br>
</div>Well you've kind of got to agree on a common format anyway haven't you<br>
if the information is going to come from metadata ? But I get your other points.<br></blockquote><div><br></div><div>We do have to define a format, but because we only implement it once if we do it at the Nova level I hope that there will be much more pragmatism than if we had to get the configuration cabal to agree. We can implement the format, and if consumers want the functionality that's the format they must parse :-)</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><span style="color:rgb(34,34,34)"> I'd just like to</span><br></div>
see it separate from the existing metadata blob, and on an opt-in basis</blockquote><div><br></div><div>Separate: is peers.json enough? I'm not sure I'm understanding you here.</div><div><br></div><div>Opt-in: IMHO, the danger of our OpenStack everything-is-optional-and-configurable approach is that we end up in a scenario where nothing is consistent and so nothing works "out of the box". I'd much rather hash-out an agreement about what is safe to share, even if that is just IPs, and then get to the point where it is globally enabled. Would you be OK with it if it was just a list of IPs?</div>
<div><br></div><div>Justin</div></div><br></div></div>