<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Justin,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">My though process is to go back to basics. To perform discovery there is no getting away from the fact that you have to start with a well-known address that
your peers can access on the network. The second part is a service/protocol accessible at that address that can perform the discovery. So the questions are: what well-known addresses can I reach? And is that a suitable place to implement the service/protocol.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The metadata service is different to the others in that it can be accessed without credentials (correct me if I’m wrong), so it is the only possibility out
of the openstack services if you do not want to have credentials on the peer instances. If that is not the case then the other services are options. All services require security groups and/or networks to be configured appropriately to access them.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">(Yes, the question “can all instances access the same metadata service” did really mean are they all local. Sorry for being unclear. But I think your answer
is yes, they are, right?)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Implementing the peer discovery in the instances themselves requires some kind of multicast or knowing a list of addresses to try. In both cases either the
actual addresses or some name resolved through a naming service would do. Whatever is starting your instances does have access to at least nova, so it can find out if there are any running instances and what their addresses are. These could be used as the
addresses they try first. These are the way that internet p2p services work and they work in the cloud.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">So there are options. The metadata service is a good place in terms of accessibility, but may not be for other reasons. In particular, the lack of credentials
relates to the fact it is only allowed to see its own information. Making that more dynamic and including information about other things in the system might change the security model slightly. Secondly, is it the purpose of the metadata server to do this job?
That’s more a matter of choice.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Personally, I think no, this is not the right place.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Paul.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Justin Santa Barbara [mailto:justin@fathomdb.com]
<br>
<b>Sent:</b> 24 January 2014 21:01<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [Nova] bp proposal: discovery of peer instances through metadata service<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">Murray, Paul (HP Cloud Services) wrote:<o:p></o:p></p>
</div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Multicast is not generally used over the internet, so the comment about removing multicast is not
really justified, and any of the approaches that work there could be used. </span>
<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I think multicast/broadcast is commonly used 'behind the firewall', but I'm happy to hear of any other alternatives that you would recommend - particularly if they can work on the cloud!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I agree that the metadata service is a sensible alternative. Do you imagine your instances all having
access to the same metadata service? Is there something more generic and not tied to the architecture of a single openstack deployment?</span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
Not sure I understand - doesn't every Nova instance has access to the metadata service, and they all connect to the same back-end database? Has anyone not deployed the metadata service? It is not cross-region / cross-provider - is that what you mean? In
terms of implementation (<a href="https://review.openstack.org/#/c/68825/">https://review.openstack.org/#/c/68825/</a>) it is supposed to be the same as if you had done a list-instances call on the API provider. I know there's been talk of federation here;
when this happens it would be awesome to have a cross-provider view (optionally, probably).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Although this is a simple example, it is also the first of quite a lot of useful primitives that
are commonly provided by configuration services. As it is possible to do what you want by other means (including using an implementation that has multicast within subnets – I’m sure neutron does actually have this), it seems that this makes less of a special
case and rather a requirement for a more general notification service?</span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I don't see any other solution offering as easy a solution for users (either the developer of the application or the person that launches the instances). If every instance had an automatic keystone token/trust with read-only access to
its own project, that would be great. If Heat intercepted every Nova call and added metadata, that would be great. If Marconi offered every instance a 'broadcast' queue where it could reach all its peers, and we had a Keystone trust for that, that would
be great. But, those are all 12 month projects, and even if you built them and they were awesome they still wouldn't get deployed on all the major clouds, so I _still_ couldn't rely on them as an application developer.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">My hope is to find something that every cloud can be comfortable deploying, that solves discovery just as broadcast/multicast solves it on typical LANs. It may be that anything other than IP addresses will make e.g. HP public cloud uncomfortable;
if so then I'll tweak it to just be IPs. Finding an acceptable solution for everyone is the most important thing to me. I am very open to any alternatives that will actually get deployed!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">One idea I had: I could return a flat list of IPs, as JSON objects:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">{ ip: '1.2.3.4' },<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">{ ip: '1.2.3.5' },<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">{ ip: '1.2.3.6' }<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If e.g. it turns out that security groups are really important, then we can just pop the extra attribute into the same data format without breaking the API:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">{ ip: '1.2.3.4', security_groups: [ 'sg1', 'sg2' ] }<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Justin<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>