<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div>Mark,</div>
<div><br>
</div>
<div>This is exactly the kind of discussion I was hoping for during Austin. I agree with pretty much all your statements. I think if Kolla can define what it would expect in the inventory provided by a bare metal provisioner, and we can make an ABI around
that, then this becomes a lot more operator friendly. I kinda hoped the discussion would start with that definition, and the delve into individual bare metal tools after that.</div>
<div><br>
</div>
<div>To add the discussion of looking a little deeper at the deployment tools: we use cobbler and have containerized it in the "Kolla way". We run TFTP and HTTP in their own containers. Cobblerd and DHCP had to be in the same container, only b/c cobbler expects
to issue "systemctl restart isc-dhcpd-server" command when it changes the DHCP config. If either cobbler or isc-dhcp could handle this is a more graceful manner, then there wouldn't be any problem putting them each in their own container. We share volumes
between the containers, and the cobblerd container runs supervisord. Cobbler has an API using xmlrpc which we utilize for system definition. It also can provide an ansible inventory, although I haven't played with that feature. I know cobbler doesn't do
the new shiny image based deployment, but for us its feature-mature, steady, and reliable.</div>
<div><br>
</div>
<div>I'd love to hear from other folks about their journey with bare metal deployment with Kolla.</div>
<div><br>
</div>
<div>Thx,</div>
<div>britt</div>
<div>
<div id="MAC_OUTLOOK_SIGNATURE"></div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Mark Casey <<a href="mailto:markcasey@pointofrental.com">markcasey@pointofrental.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Monday, May 9, 2016 at 6:48 PM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [kolla] [bifrost] bifrost container.<br>
</div>
<div><br>
</div>
<div>
<div bgcolor="#FFFFFF" text="#000000">I'm not sure if it is necessary to write up or provide support on how to use more than one deployment tool, but I think any work that inadvertently makes it harder for an operator to use their own existing deployment infrastructure
could run some people off.<br>
<br>
Regarding "deploy a VM to deploy bifrost to deploy bare metal", I suspect that situation will not be unique to bifrost. At the moment I'm using MAAS and it has a hard dependency on Upstart for init up until around Ubuntu Trusty and then was ported to systemd
in Wily. I do not think you can just switch to another init daemon or run it under supervisord without significant work. I was not even able to get the maas package to install during a docker build because it couldn't communicate with the init system it wanted.
In addition, for any deployment tool that enrolls/deploys via PXE the tool may also require accommodations when being containerized simply because this whole topic is fairly low in the stack of abstractions. For example I'm not sure whether any of these tools
running in a container would respond to a new bare metal host's initial DHCP broadcast without --net=host or similar consideration.<br>
<br>
As long as the most common deployment option in Kolla is Ansible, making deployment tools pluggable is fairly easy to solve. MAAS and bifrost both have inventory scripts that can provide dynamic inventory to kolla-ansible while still pulling Kolla's child groups
from the multinode inventory file. Another common pattern could be for a given deployment tool to template out a new (static) multinode inventory and then we just append Kolla's groups to the file before calling kolla-ansible. The problem, to me, becomes in
getting every other option (k8s, puppet, etc.) to work similarly. Perhaps you just state that each implementation must be pluggable to various deployment tools and let people that know their respective tool handle the how.(?)<br>
<br>
Currently I am running MAAS inside a Vagrant box to retain some of the immutability and easy "create/destroy" workflow that having it containerized would offer. It works very well and, assuming nothing else was running on the underlying deployment host, I'd
have no issue running it in prod that way even with the Vagrant layer.<br>
<br>
Thank you,<br>
Mark<br>
<br>
<div class="moz-cite-prefix">On 5/9/2016 4:52 PM, Britt Houser (bhouser) wrote:<br>
</div>
<blockquote cite="mid:73A5E80C-D191-4874-B1F4-A11D3786FF44@cisco.com" type="cite">
<div>
<div>
<div>Are we (as the Kolla community) open to other bare metal provisioners? The austin discussion was titled generic bare metal, but very quickly turned into bifrost-only discourse. The initial survey showed cobbler/maas/OoO as alternatives people use today.
So if the bifrost strategy is, "deploy a VM to deploy bifrost to deploy bare metal" and will cleaned up later, then maybe its time to take a deeper look at the other deployment tools and see if they are a better fit?</div>
<div><br>
</div>
<div>Thx,</div>
<div>britt</div>
<div></div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:12pt;
text-align:left; color:black; BORDER-BOTTOM: medium none;
BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT:
0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;
BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>"Steven Dake (stdake)" <<a moz-do-not-send="true" href="mailto:stdake@cisco.com">stdake@cisco.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org"></a><a class="moz-txt-link-abbreviated" href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Monday, May 9, 2016 at 5:41 PM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org"></a><a class="moz-txt-link-abbreviated" href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [kolla] [bifrost] bifrost container.<br>
</div>
<div><br>
</div>
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space; color: rgb(0, 0, 0);
font-size: 14px; font-family: Calibri, sans-serif;">
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM: medium
none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in;
PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP:
#b5c4df 1pt solid; BORDER-RIGHT: medium none;
PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Devananda van der Veen <<a moz-do-not-send="true" href="mailto:devananda.vdv@gmail.com">devananda.vdv@gmail.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org"></a><a class="moz-txt-link-abbreviated" href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Monday, May 9, 2016 at 1:12 PM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org"></a><a class="moz-txt-link-abbreviated" href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [kolla] [bifrost] bifrost container.<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;
MARGIN:0 0 0 5;">
<div>
<div>
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, May 6, 2016 at 10:56 AM, Steven Dake (stdake) <span dir="ltr">
<<a moz-do-not-send="true" href="mailto:stdake@cisco.com" target="_blank">stdake@cisco.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>Sean,</div>
<div><br>
</div>
<div>Thanks for taking this on :) I didn't know you had such an AR :)</div>
<div><br>
</div>
<span>
<div style="font-family:Calibri;font-size:11pt;text-align:left;color:black;BORDER-BOTTOM:medium
none;BORDER-LEFT:medium
none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df
1pt solid;BORDER-RIGHT:medium
none;PADDING-TOP:3pt">
<span style="font-weight:bold">From: </span>"Mooney, Sean K" <<a moz-do-not-send="true" href="mailto:sean.k.mooney@intel.com" target="_blank">sean.k.mooney@intel.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org" target="_blank"></a><a class="moz-txt-link-abbreviated" href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Friday, May 6, 2016 at 10:14 AM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a moz-do-not-send="true" href="mailto:openstack-dev@lists.openstack.org" target="_blank"></a><a class="moz-txt-link-abbreviated" href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>[openstack-dev] [kolla] [bifrost] bifrost container.<br>
</div>
<span class="">
<div><br>
</div>
<blockquote style="BORDER-LEFT:#b5c4df
5 solid;PADDING:0 0 0 5;MARGIN:0 0 0
5">
<div>
<div link="#0563C1" vlink="#954F72" lang="EN-US">
<div>
<p class="MsoNormal">Hi everyone.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Following up on my AR from the kolla host repository session</p>
<p class="MsoNormal"><a moz-do-not-send="true" href="https://etherpad.openstack.org/p/kolla-newton-summit-kolla-kolla-host-repo" target="_blank"></a><a class="moz-txt-link-freetext" href="https://etherpad.openstack.org/p/kolla-newton-summit-kolla-kolla-host-repo">https://etherpad.openstack.org/p/kolla-newton-summit-kolla-kolla-host-repo</a></p>
<p class="MsoNormal">I started working on creating a kolla bifrost container.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Are some initial success it have hit a roadblock with the current install playbook provided by bifrost.</p>
<p class="MsoNormal">In particular the install playbook both installs the ironic dependencies and configure and runs the services.</p>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</blockquote>
</span></span>
<div><br>
</div>
<div>What I'd do here is ignore the install playbook and duplicate what it installs. We don't want to install at run time, we want to install at build time. You weren't clear if that is what your doing.</div>
</div>
</blockquote>
<div><br>
</div>
<div>That's going to be quite a bit of work. The bifrost-install playbook does a lot more than just install the ironic services and a few system packages; it also installs rabbit, mysql, nginx, dnsmasq *and* configures all of these in a very specific way. Re-inventing
all of this is basically re-inventing Bifrost.</div>
<div> <br>
</div>
<div> </div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>Sean's latest proposal was splitting this one operation into three smaller decomposed steps.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;
MARGIN:0 0 0 5;">
<div>
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>The reason we would ignore the install playbook is because it runs the services. We need to run the services in a different way.
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Do you really need to run them in a different way? If it's just a matter of "use a different init system", I wonder how easily that could be accomodated within the Bifrost project itself.... If there's another reason, please elaborate.</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>To run in a container, we cannot use systemd. This leaves us with supervisord, which certainly can and should be done in the context of upstream bifrost.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;
MARGIN:0 0 0 5;">
<div>
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div> This will (as we discussed at ODS) be a fat container on the underlord cloud – which I guess is ok. I'd recommend not using systemd, as that will break systemd systems badly. Instead use a different init system, such as supervisord.</div>
<span class="">
<div><br>
</div>
<span>
<blockquote style="BORDER-LEFT:#b5c4df
5 solid;PADDING:0 0 0 5;MARGIN:0 0 0
5">
<div>
<div link="#0563C1" vlink="#954F72" lang="EN-US">
<div>
<p class="MsoNormal">The installation of ironic and its dependencies would not be a problem but the ansible service module is not cable able of starting the</p>
<p class="MsoNormal">Infrastructure services (mysql,rabbit …) without a running init system which is not present during the docker build.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">When I created a biforst container in the past is spawned a Ubuntu upstart container then docker exec into the container and ran</p>
<p class="MsoNormal">Bifrost install script. This works because the init system is running and the service module could test and start the relevant services.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">This leave me with 3 paths forward.</p>
<p class="MsoNormal"> </p>
<p><span>1.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>I can continue to try and make the bifrost install script work with the kolla build system by using sed to modify the install playbook or try start systemd during the docker build.</p>
<p><span>2.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>I can use the kolla build system to build only part of the image</p>
<p style="margin-left:72.0pt"><span>a.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span> the bifrost-base image would be build with the kolla build system without running the bifrost playbook. This<br>
would allow the existing allow the existing features of the build system such as adding headers/footers to be used.</p>
<p style="margin-left:72.0pt"><span>b.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>After the base image is built by kolla I can spawn an instance of bifrost-base with systemd running
</p>
<p style="margin-left:72.0pt"><span>c.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>I can then connect to this running container and run the bifrost install script unmodified.</p>
<p style="margin-left:72.0pt"><span>d.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>Once it is finished I can stop the container and export it to an image “bifros-postinstall”.
</p>
<p style="margin-left:72.0pt"><span>e.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>This can either be used directly (fat container) or as the base image for other container that run each of the ironic services (thin containers)</p>
<p><span>3.<span style="font-style:normal;font-weight:normal;font-size:7pt;line-height:normal;font-family:'Times
New Roman'">
</span></span>I can skip the kolla build system entirely and create a script/playbook that will build the bifrost container similar to 2.</p>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
</span>
<div>4.</div>
<div>Make a supervisord set of init scripts and make the docker file do what it was intended – install the files. This is kind of a mashup of your 1-3 ideas. Good thinking :)</div>
<span class=""><span>
<blockquote style="BORDER-LEFT:#b5c4df
5 solid;PADDING:0 0 0 5;MARGIN:0 0 0
5">
<div>
<div link="#0563C1" vlink="#954F72" lang="EN-US">
<div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">While option 1 would fully use the kolla build system It is my least favorite as it is both hacky and complicated to make work.
</p>
<p class="MsoNormal">Docker really was not designed to run systemd as part of docker build.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">For option 2 and 3 I can provide a single playbook/script that will fully automate the build but the real question I have</p>
<p class="MsoNormal">Is should I use the kolla build system to make the base image or not.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">If anyone else has suggestion on how I can progress please let me know but currently I am leaning towards option 2.
</p>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
</span>
<div>If you have questions about my suggestion to use supervisord, hit me up on IRC. Ideally we would also contribute these init scripts back into bifrost code base assuming they want them, which I think they would. Nobody will run systemd in a container,
and we all have an interest in seeing BiFrost as the standard bare metal deployment model inside or outside of containers.</div>
<div><br>
</div>
<div>Regards</div>
<div>-steve</div>
<span class="">
<div><br>
</div>
<span>
<blockquote style="BORDER-LEFT:#b5c4df
5 solid;PADDING:0 0 0 5;MARGIN:0 0 0
5">
<div>
<div link="#0563C1" vlink="#954F72" lang="EN-US">
<div>
<p class="MsoNormal">The only other option I see would be to not use a container and either install biforst on the host or in a vm.</p>
</div>
</div>
</div>
</blockquote>
</span></span>
<div>GROAN – one advantage containers provide us is not mucking up the host OS with a bajillion dependencies. I'd like to keep that part of Kolla intact :)</div>
</div>
</blockquote>
<div><br>
</div>
<div>Right - don't install it on the host, but what's the problem with running it in a VM?</div>
<div><br>
</div>
<div>FWIW, I already run Bifrost quite successfully in a VM in each of my environments.</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span>
<div><br>
</div>
<div>There isn't a super specific problem with running it in a VM other than Kolla is about containers not VMs. OpenStack can obviously be run in a VM – our major reason for wanting containers is upgradability which Vms don't offer atomically.</div>
<div><br>
</div>
<div>That said, we could run in a VM initially and over time port to run in a container. What we are after long term is a container–based approach to bifrost in upstream bifrost, not replicating or duplicating a bunch of work.</div>
<div><br>
</div>
<div>I believe Sean's approach of splitting out the 3 separate steps makes logical sense (to me) in the sense that the one major installation step is broken into the separate build & deploy steps that Kolla uses.</div>
<div><br>
</div>
<div>Hope that helps</div>
<div><br>
</div>
<div>Regards</div>
<div>-steve</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;
MARGIN:0 0 0 5;">
<div>
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div><br>
</div>
<div>--Deva</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span></div>
</div>
</span><br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></pre>
</blockquote>
<br>
</div>
</div>
</span>
</body>
</html>