<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div><br>
</div>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Andrew Melton <<a href="mailto:andrew.melton@RACKSPACE.COM">andrew.melton@RACKSPACE.COM</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, September 17, 2015 at 12:06 PM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [magnum] Discovery<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div><style type="text/css" style="display:none"><!-- p { margin-top: 0px; margin-bottom: 0px; }--></style>
<div dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Hey Daneyon,</p>
<p><br>
</p>
<p>I'm fairly partial towards #2 as well. Though, I'm wondering if it's possible to take it a step further. Could we run etcd in each Bay without using the public discovery endpoint? And then, configure Swarm to simply use the internal ectd as it's discovery
mechanism? This could cut one of our external service dependencies and make it easier to run Magnum is an environment with locked down public internet access.</p>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>Thanks for your feedback. #2 was the preferred direction of the Magnum Networking Subteam as well. Therefore, I have started working on [1] to move this option forward. As part of this effort, I am slightly refactoring the Swarm heat templates to more
closely align them with the k8s templates. Until tcammann completes the larger template refactor [2], I think this will help dev’s more easily implement features across all bay types, distro’s, etc..</div>
<div><br>
</div>
<div>We can have etcd and Swarm use a local discovery backend. I have filed bp [3] to establish this effort. As a first step towards [3], I will modify Swarm to use etcd for discovery. However, etcd will still use public discovery for [1]. Either myself or
someone from the community will need to attack etcd local discovery as a follow-on. For the longer-term, we may want to consider exposing a --discovery-backend attribute and optionally pass labels to allow users to modify default configurations of the —discovery-backend.</div>
<div><br>
</div>
<div>[1] <a href="https://review.openstack.org/#/c/224367/">https://review.openstack.org/#/c/224367/</a></div>
<div>[2] <a href="https://blueprints.launchpad.net/magnum/+spec/generate-heat-templates">https://blueprints.launchpad.net/magnum/+spec/generate-heat-templates</a></div>
<div>[3] <a href="https://blueprints.launchpad.net/magnum/+spec/bay-type-discovery-options">https://blueprints.launchpad.net/magnum/+spec/bay-type-discovery-options</a></div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>
<div dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p><br>
</p>
<p><br>
</p>
<p>Anyways, I think #2 could be a good start that we could iterate on later if need be.<br>
</p>
<p><br>
</p>
<p>--Andrew<br>
</p>
<p><br>
</p>
<div style="word-wrap:break-word; color:rgb(0,0,0); font-size:14px; font-family:Calibri,sans-serif">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri,sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Daneyon Hansen (danehans) <<a href="mailto:danehans@cisco.com">danehans@cisco.com</a>><br>
<b>Sent:</b> Wednesday, September 16, 2015 11:26 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> [openstack-dev] [magnum] Discovery</font>
<div> </div>
</div>
<div>
<div>All,</div>
<div><br>
</div>
<div>While implementing the flannel --network-driver for swarm, I have come across an issue that requires feedback from the community. Here is the breakdown of the issue:</div>
<ol>
<li>Flannel [1] requires etcd to store network configuration. Meeting this requirement is simple for the kubernetes bay types since kubernetes requires etcd.<br>
</li><li>A discovery process is needed for bootstrapping etcd. Magnum implements the public discovery option [2].</li><li>A discovery process is also required to bootstrap a swarm bay type. Again, Magnum implements a publicly hosted (Docker Hub) option [3].</li><li>Magnum API exposes the discovery_url attribute that is leveraged by swarm and etcd discovery.</li><li>Etcd can not be implemented in swarm because discovery_url is associated to swarm’s discovery process and not etcd.</li></ol>
<div>Here are a few options on how to overcome this obstacle:</div>
<ol>
<li>Make the discovery_url more specific, for example etcd_discovery_url <i>and </i>
swarm_discovery_url. However, this option would needlessly expose both discovery url’s to all bay types.</li><li>Swarm supports etcd as a discovery backend. This would mean discovery is similar for both bay types. With both bay types using the same mechanism for discovery, it will be easier to provide a private discovery option in the future.</li><li>Do not support flannel as a network-driver for k8s bay types. This would require adding support for a different driver that supports multi-host networking such as libnetwork. Note: libnetwork is only implemented in the Docker experimental release: <a href="https://github.com/docker/docker/tree/master/experimental">https://github.com/docker/docker/tree/master/experimental</a>.</li></ol>
<div>I lean towards #2 but their may be other options, so feel free to share your thoughts. I would like to obtain feedback from the community before proceeding in a particular direction.</div>
<div><br>
</div>
<div>[1] <a href="https://github.com/coreos/flannel">https://github.com/coreos/flannel</a></div>
<div><u>[2] </u><a href="https://github.com/coreos/etcd/blob/master/Documentation/discovery_protocol.md">https://github.com/coreos/etcd/blob/master/Documentation/discovery_protocol.md</a></div>
<div><u>[3] </u><a href="https://docs.docker.com/swarm/discovery/">https://docs.docker.com/swarm/discovery/</a></div>
<div><br>
</div>
<div>
<div>Regards,</div>
<div>Daneyon Hansen</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
</body>
</html>