<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">+1.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Mark Goddard <mark@stackhpc.com><br>
<b>Reply-To: </b>"OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org><br>
<b>Date: </b>Friday, April 6, 2018 at 11:41 AM<br>
<b>To: </b>"OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org><br>
<b>Subject: </b>Re: [openstack-dev] [kolla] [tripleo] On moving start scripts out of Kolla images<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><a name="_MailOriginalBody">One benefit of the kolla API that I've not seen mentioned yet (sorry if I missed it) is that you can change files on the host without affecting the running container. Bind mounts don't have this property. This
is handy for reconfiguration/upgrade operations, where we write out a new set of config before recreating/restarting the container. COPY_ONCE is the king of immutable here, but even for COPY_ALWAYS, this works as long as the container doesn't restart while
the config files are being written. <o:p></o:p></a></p>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">Mark<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">On 5 April 2018 at 21:41, Michał Jastrzębski <</span><a href="mailto:inc007@gmail.com" target="_blank"><span style="mso-bookmark:_MailOriginalBody">inc007@gmail.com</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">>
wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody">So I'll re-iterate comment which I made in BCN. In previous thread we<br>
praised how Kolla provided stable API for images, and I agree that it<br>
was great design choice (to provide stable API, not necessarily how<br>
API looks), and this change would break it. So *if* we decide to do<br>
it, we need to follow deprecation, that means we could deprecate these<br>
files in this release and start removing them in next.<br>
<br>
Support for LOCI in kolla-ansible is good thing, but I don't think<br>
changing Kolla image API is required for that. LOCI provides base<br>
image arument, so we could simply create base-image with all the<br>
extended-start and set-config mechanisms and some shim to source<br>
extended-start script that belongs to particular container. We will<br>
need kolla layer image anyway because set_config is there to stay (as<br>
Martin pointed out it's valuable tool fixing real issue and it's used<br>
by more projects than just kolla-ansible). We could add another script<br>
that would look like extended_start.sh -> source<br>
$CONTAINER_NAME-extended-start.sh and copy all kolla's extended start<br>
scripts to dir with proper naming (I believe this is solution that Sam<br>
came up with shortly after BCN). This is purely techincal and not that<br>
hard to do, much quicker and easier than deprecating API...<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><br>
On 5 April 2018 at 12:28, Martin André <</span><a href="mailto:m.andre@redhat.com"><span style="mso-bookmark:_MailOriginalBody">m.andre@redhat.com</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">> wrote:<br>
> On Thu, Apr 5, 2018 at 2:16 PM, Paul Bourke <</span><a href="mailto:paul.bourke@oracle.com"><span style="mso-bookmark:_MailOriginalBody">paul.bourke@oracle.com</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">>
wrote:<br>
>> Hi all,<br>
>><br>
>> This mail is to serve as a follow on to the discussion during yesterday's<br>
>> team meeting[4], which was regarding the desire to move start scripts out of<br>
>> the kolla images [0]. There's a few factors at play, and it may well be best<br>
>> left to discuss in person at the summit in May, but hopefully we can get at<br>
>> least some of this hashed out before then.<br>
>><br>
>> I'll start by summarising why I think this is a good idea, and then attempt<br>
>> to address some of the concerns that have come up since.<br>
>><br>
>> First off, to be frank, this is effort is driven by wanting to add support<br>
>> for loci images[1] in kolla-ansible. I think it would be unreasonable for<br>
>> anyone to argue this is a bad objective to have, loci images have very<br>
>> obvious benefits over what we have in Kolla today. I'm not looking to drop<br>
>> support for Kolla images at all, I simply want to continue decoupling things<br>
>> to the point where operators can pick and choose what works best for them.<br>
>> Stemming from this, I think moving these scripts out of the images provides<br>
>> a clear benefit to our consumers, both users of kolla and third parties such<br>
>> as triple-o. Let me explain why.<br>
><br>
> It's still very obscure to me how removing the scripts from kolla<br>
> images will benefit consumers. If the reason is that you want to<br>
> re-use them in other, non-kolla images, I believe we should package<br>
> the scripts. I've left some comments in your spec review.<br>
><br>
>> Normally, to run a docker image, a user will do 'docker run<br>
>> helloworld:latest'. In any non trivial application, config needs to be<br>
>> provided. In the vast majority of cases this is either provided via a bind<br>
>> mount (docker run -v hello.conf:/etc/hello.conf helloworld:latest), or via<br>
>> environment variables (docker run --env HELLO=paul helloworld:latest). This<br>
>> is all bog standard stuff, something anyone who's spent an hour learning<br>
>> docker can understand.<br>
>><br>
>> Now, lets say someone wants to try out OpenStack with Docker, and they look<br>
>> at Kolla. First off they have to look at something called set_configs.py[2]<br>
>> - over 400 lines of Python. Next they need to understand what that script<br>
>> consumes, config.json [3]. The only reference for config.json is the files<br>
>> that live in kolla-ansible, a mass of jinja and assumptions about how the<br>
>> service will be run. Next, they need to figure out how to bind mount the<br>
>> config files and config.json into the container in a way that can be<br>
>> consumed by set_configs.py (which by the way, requires the base kolla image<br>
>> in all cases). This is only for the config. For the service start up<br>
>> command, this need to also be provided in config.json. This command is then<br>
>> parsed out and written to a location in the image, which is consumed by a<br>
>> series of start/extend start shell scripts. Kolla is *unique* in this<br>
>> regard, no other project in the container world is interfacing with images<br>
>> in this way. Being a snowflake in this regard is not a good thing. I'm still<br>
>> waiting to hear from a real world operator who would prefer to spend time<br>
>> learning the above to doing:<br>
><br>
> You're pointing a very real documentation issue. I've mentioned in the<br>
> other kolla thread that I have a stub for the kolla API documentation.<br>
> I'll push a patch for what I have and we can iterate on that.<br>
><br>
>> docker run -v /etc/keystone:/etc/keystone keystone:latest --entrypoint<br>
>> /usr/bin/keystone [args]<br>
>><br>
>> This is the Docker API, it's easy to understand and pretty much the standard<br>
>> at this point.<br>
><br>
> Sure, using the docker API works for simpler cases, not too<br>
> surprisingly once you start doing more funky things with your<br>
> containers you're quickly reach the docker API limitations. That's<br>
> when the kolla API comes in handy.<br>
> See for example this recent patch<br>
> </span><a href="https://review.openstack.org/#/c/556673/" target="_blank"><span style="mso-bookmark:_MailOriginalBody">https://review.openstack.org/#/c/556673/</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">
where we needed to change<br>
> some file permission to the uid/gid of the user inside the container.<br>
><br>
> The first iteration basically used the docker API and started an<br>
> additional container to fix the permissions:<br>
><br>
> docker run -v<br>
> /etc/pki/tls/certs/neutron.crt:/etc/pki/tls/certs/neutron.crt:rw \<br>
> -v /etc/pki/tls/private/neutron.key:/etc/pki/tls/private/neutron.key:rw<br>
> \<br>
> neutron_image \<br>
> /bin/bash -c 'chown neutron:neutron<br>
> /etc/pki/tls/certs/neutron.crt; chown neutron:neutron<br>
> /etc/pki/tls/private/neutron.key'<br>
><br>
> You'll agree this is not the most obvious. And it had a nasty side<br>
> effect that is changes the permissions of the files _on the host_.<br>
> While using kolla API we could simply add to our config.json:<br>
><br>
> - path: /etc/pki/tls/certs/neutron.crt<br>
> owner: neutron:neutron<br>
> - path: /etc/pki/tls/private/neutron.key<br>
> owner: neutron:neutron<br>
><br>
>> The other argument is that this removes the possibility for immutable<br>
>> infrastructure. The concern is, with the new approach, a rookie operator<br>
>> will modify one of the start scripts - resulting in uncertainty that what<br>
>> was first deployed matches what is currently running. But with the way Kolla<br>
>> is now, an operator can still do this! They can restart containers with a<br>
>> custom entrypoint or additional bind mounts, they can exec in and change<br>
>> config files, etc. etc. Kolla containers have never been immutable and we're<br>
>> bending over backwards to artificially try and make this the case. We cant<br>
>> protect a bad or inexperienced operator from shooting themselves in the<br>
>> foot, there are better ways of doing so. If/when Docker or the upstream<br>
>> container world solves this problem, it would then make sense for Kolla to<br>
>> follow suit.<br>
>><br>
>> On the face of it, what the spec proposes is a simple change, it should not<br>
>> radically pull the carpet out under people, or even change the way<br>
>> kolla-ansible works in the near term. If consumers such as tripleo or other<br>
>> parties feel it would in fact do so please do let me know and we can discuss<br>
>> and mitigate these problems.<br>
><br>
> TripleO uses these scripts extensively, we certainly do not want to<br>
> see them go away from kolla images.<br>
><br>
> Martin<br>
><br>
>> Cheers,<br>
>> -Paul<br>
>><br>
>> [0] </span><a href="https://review.openstack.org/#/c/550958/" target="_blank"><span style="mso-bookmark:_MailOriginalBody">https://review.openstack.org/#/c/550958/</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
>> [1] </span><a href="https://github.com/openstack/loci" target="_blank"><span style="mso-bookmark:_MailOriginalBody">https://github.com/openstack/loci</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
>> [2]<br>
>> </span><a href="https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py" target="_blank"><span style="mso-bookmark:_MailOriginalBody">https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
>> [3]<br>
>> </span><a href="https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/templates/keystone.json.j2" target="_blank"><span style="mso-bookmark:_MailOriginalBody">https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/templates/keystone.json.j2</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
>> [4]<br>
>> </span><a href="http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-04-04-16.00.log.txt" target="_blank"><span style="mso-bookmark:_MailOriginalBody">http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-04-04-16.00.log.txt</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
>><br>
>> __________________________________________________________________________<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe: </span><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank"><span style="mso-bookmark:_MailOriginalBody">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
>> </span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><span style="mso-bookmark:_MailOriginalBody">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: </span><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank"><span style="mso-bookmark:_MailOriginalBody">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
> </span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><span style="mso-bookmark:_MailOriginalBody">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: </span><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank"><span style="mso-bookmark:_MailOriginalBody">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
</span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><span style="mso-bookmark:_MailOriginalBody">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
</div>
</body>
</html>