<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Like Doug said Amphora suppose to be a black box. It suppose to get some data - like info in /etc/defaults and do everything inside on its own.
<div class="">Everyone will be able to prepare his own implementation of this image without mixing things between each other.</div>
<div class=""> <br class="">
<div class="">
<div class="">
<div class="">Lubosz Kosnik</div>
<div class="">Cloud Software Engineer OSIC</div>
<div class=""><a href="mailto:lubosz.kosnik@intel.com" class="">lubosz.kosnik@intel.com</a></div>
</div>
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jun 29, 2016, at 3:17 PM, Gregory Haynes <<a href="mailto:greg@greghaynes.net" class="">greg@greghaynes.net</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<title class=""></title>
<div class="">
<div class="">On Wed, Jun 29, 2016, at 02:18 PM, Nir Magnezi wrote:<br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">Hi Greg,<br class="">
</div>
<div class=""> </div>
<div class="">
<div class="">Thanks for the replay, comments inline.<br class="">
</div>
<div class="">
<div class=""> </div>
<div defang_data-gmailquote="yes" class="">
<div class="">On Wed, Jun 29, 2016 at 9:59 PM, Gregory Haynes <span dir="ltr" class="">
<<a href="mailto:greg@greghaynes.net" class="">greg@greghaynes.net</a>></span> wrote:<br class="">
</div>
<blockquote style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;" defang_data-gmailquote="yes" class="">
<div class=""><u class=""></u><br class="">
</div>
<div class="">
<div class="">
<div class="">
<div class="">On Wed, Jun 29, 2016, at 10:26 AM, Nir Magnezi wrote:<br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">Hello,<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">Lately, I've been working on a fix[1] for the amhpora-agent, which currently only support Debian based flavors such as Ubuntu.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">The main Issues here:<br class="">
</div>
<div class="">1. NIC hot plugs: Ubuntu's ethX.cfg files looks different from ifcfg-ethX files which are accepted in Linux flavors such a RHEL, CentOS and Fedora, read more in the fix commit msg.<br class="">
</div>
<div class="">2. The usage of Flavor specific features such as 'upstart'.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">I would like to have a discussion about the second bullet mentioned above.<br class="">
</div>
<div class="">Due to the fact that in Octavia the loadbalancer runs inside of an instance (Amphora), There are few actions that need to take place in the Amphora instance boot process:<br class="">
</div>
<div class="">a. namespace and NIC created.<br class="">
</div>
<div class="">b. amphora agent starts<br class="">
</div>
<div class="">c. haproxy (and possibly keepalived) start<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">
<div class="">The Amphora-agent leverages[2] the capabilities of 'upstart' to make that happen, which is a bit problematic if we wish it to work on other flavors.<br class="">
</div>
</div>
<div class="">The default cloud image for Amphora today is Ubuntu, yet there are few more options[3] such as CentOS and Fedora.<br class="">
</div>
<div class="">Unlike the Ubuntu base image, which uses 'sysvinit', The latter two flavors use 'systemd'.<br class="">
</div>
<div class="">This creates incompatibility with the jinja2[4][5] templates used by the agent.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">The way I see it there are two possible solutions for this:<br class="">
</div>
<div class="">1. Include a systemd equivalent in the fix[1] that will essentially duplicate the functionality mentioned above and work in the other flavors.<br class="">
</div>
<div class="">2. Have a the amphora agent be the only binary that needs to be configured to start upon boot, and that agent will take care of plugging namespaces and NICs and also spawning needs processes. This is how it is done in lbaas and l3 agents.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">While the latter solution looks like a more "clean" design, the trade-off here is a bigger change to the amphora agent. <br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">[1] <a href="https://review.openstack.org/#/c/331841/" class="">https://review.openstack.org/#/c/331841/</a><br class="">
</div>
<div class="">
<div class="">[2] <a href="https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/listener.py#L128" class="">
https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/listener.py#L128</a><br class="">
</div>
</div>
<div class="">[3] <a href="https://github.com/openstack/octavia/blob/master/diskimage-create/diskimage-create.sh#L27" class="">https://github.com/openstack/octavia/blob/master/diskimage-create/diskimage-create.sh#L27</a><br class="">
</div>
<div class="">[4] <a href="https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/templates/upstart.conf.j2" class="">https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/templates/upstart.conf.j2</a><br class="">
</div>
<div class="">[5] <a href="https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/templates/sysvinit.conf.j2" class="">https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/templates/sysvinit.conf.j2</a><br class="">
</div>
<div class=""> <br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">Thanks,<br class="">
</div>
<div class="">Nir<br class="">
</div>
</div>
</blockquote>
<div class=""> <br class="">
</div>
</div>
</div>
<div class="">I have an alternative suggestion - Maybe we shouldn't be templating out the init scripts? What we are effectively doing here is code-gen which leads to problems exactly like this, and fixing it with more code gen actually makes the problem more
difficult.<br class="">
</div>
<div class=""> <br class="">
</div>
</div>
</blockquote>
<div class="">The incompatibility to systemd is not due to usage of templates and code generated files is a nice and useful tool to have.<br class="">
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Sure, its not a direct result, but it just shouldn't be necessary here and it makes this problem far more complicated than it needs to be. If we weren't using templating then supporting non-upstart would be as easy as creating a trivial init script
and including it in the amphora element (which only requies copying a file in to that element, done.).<br class="">
</div>
<div class=""> </div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div defang_data-gmailquote="yes" class="">
<div class=""> <br class="">
</div>
<blockquote style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;" defang_data-gmailquote="yes" class="">
<div class="">
<div class="">I see two fairly straightforward ways to not require this templating:<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">1) Use the agent to write out config for the init scripts in to /etc/defaults/amphora and have the init scripts consume that file (source variables in that file). The init script can then simply be a static file which we can even bake in to the
image directly.<br class="">
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">systemd does not use init script, which is why the current code is incompatible to the distros i mentioned. <br class="">
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Right, what I am saying is to separate out configuration from the init/upstart/systemd files and if necessary source that configuration. This is how init/upstart/systemd scripts are written for almost every application for a reason and why ubuntu
has /etc/defaults and why systemd has things like EnvironmentFile. It sounds like the second option is what were leaning towards though, in which case this isn't needed.</div>
<div class=""> </div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div defang_data-gmailquote="yes" class="">
<blockquote style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;" defang_data-gmailquote="yes" class="">
<div class="">
<div class=""> </div>
<div class=""> <br class="">
</div>
<div class="">2) Move the code which requires the templating in to another executable which the init scripts call out to. e.g. create a amphora-net-init executable that runs the same code as in the pre-up section of the upstart script. Then there is no need
for templating in the init scripts themselves (they will all simply call the same executable) and we can also do something like bake init scripts directly in to the image.<br class="">
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">I'm not sure I follow you here. but anyhow we cannot just give up the templates. how would you handle the parameters currently passed to those templates? some if them you cannot know in advance and just hardcode.<br class="">
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">That is exactly what I am trying to explain :). It seems like most of the parameters simply don't need to be (why does pidfile path need to be a templated parameter? This isn't exactly unique to this application, almost every init script on the
system uses a pidfile without this issue). The ones that absolutely require configuration (maybe amphora_nsname) can be split out of the init script either by using /etc/defaults or EnvironmentFile or by using the 2) proposed solution (remove the code requiring
this variable in to a separate executable).</div>
<div class=""> </div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div defang_data-gmailquote="yes" class="">
<blockquote style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;" defang_data-gmailquote="yes" class="">
<div class="">
<div class=""> </div>
<div class=""> <br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">My thinking is that this is only going to get more complex - what are we going to do to support the new ubuntu LTS which is systemd based? Or if folk show up with other distros that match neither? Its a lot easier to decouple the init scripts,
which are target specific, from configuration specific components and avoid this issues all together.<br class="">
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Well, there is a solution that I mentioned in my first mail. both neutron LBaaS and L3 agents take care of everything needed as far as namespaces , nics and processes. that would leave you with a single binary to activate upon boot.<br class="">
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Ah, sorry I missed that - I think having the agent 'just work' on startup sounds ideal. If that is the case then you shouldn't have any need to template out the init scripts, either and we can remove a lot of code and complexity :).</div>
<div class=""> </div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div defang_data-gmailquote="yes" class="">
<div class=""> <br class="">
</div>
<blockquote style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;" defang_data-gmailquote="yes" class="">
<div class="">
<div class=""> </div>
<div class=""> <br class="">
</div>
<div class="">HTH,<br class="">
</div>
<div class="">Greg<br class="">
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Nir <br class="">
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Cheers,<br class="">
</div>
<div class="">Greg</div>
</div>
__________________________________________________________________________<br class="">
OpenStack Development Mailing List (not for usage questions)<br class="">
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" class="">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>