<div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr">On Thu, 5 Apr 2018, 20:28 Martin André, <<a href="mailto:m.andre@redhat.com">m.andre@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Apr 5, 2018 at 2:16 PM, Paul Bourke <<a href="mailto:paul.bourke@oracle.com" target="_blank" rel="noreferrer">paul.bourke@oracle.com</a>> wrote:<br>
> Hi all,<br>
><br>
> This mail is to serve as a follow on to the discussion during yesterday's<br>
> team meeting[4], which was regarding the desire to move start scripts out of<br>
> the kolla images [0]. There's a few factors at play, and it may well be best<br>
> left to discuss in person at the summit in May, but hopefully we can get at<br>
> least some of this hashed out before then.<br>
><br>
> I'll start by summarising why I think this is a good idea, and then attempt<br>
> to address some of the concerns that have come up since.<br>
><br>
> First off, to be frank, this is effort is driven by wanting to add support<br>
> for loci images[1] in kolla-ansible. I think it would be unreasonable for<br>
> anyone to argue this is a bad objective to have, loci images have very<br>
> obvious benefits over what we have in Kolla today. I'm not looking to drop<br>
> support for Kolla images at all, I simply want to continue decoupling things<br>
> to the point where operators can pick and choose what works best for them.<br>
> Stemming from this, I think moving these scripts out of the images provides<br>
> a clear benefit to our consumers, both users of kolla and third parties such<br>
> as triple-o. Let me explain why.<br>
<br>
It's still very obscure to me how removing the scripts from kolla<br>
images will benefit consumers. If the reason is that you want to<br>
re-use them in other, non-kolla images, I believe we should package<br>
the scripts. I've left some comments in your spec review.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">+1 to extracting and packaging the kolla API. This will make it easier to test and document, allow for versioning, and make it a first class citizen rather than a file in the build context of the base image. Plus, if it really is as good as some people are arguing, then it should be shared.</div><div dir="auto"><br></div><div dir="auto">For many of the other helper scripts that get bundled into the kolla images, I can see an argument for pulling these up to the deployment layer. These could easily be moved to kolla-ansible, and added via config.json. I guess it would be useful to know whether other deployment tools (tripleo) are using any of these - if they are shared then perhaps the images are the best place for them.</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
> Normally, to run a docker image, a user will do 'docker run<br>
> helloworld:latest'. In any non trivial application, config needs to be<br>
> provided. In the vast majority of cases this is either provided via a bind<br>
> mount (docker run -v hello.conf:/etc/hello.conf helloworld:latest), or via<br>
> environment variables (docker run --env HELLO=paul helloworld:latest). This<br>
> is all bog standard stuff, something anyone who's spent an hour learning<br>
> docker can understand.<br>
><br>
> Now, lets say someone wants to try out OpenStack with Docker, and they look<br>
> at Kolla. First off they have to look at something called set_configs.py[2]<br>
> - over 400 lines of Python. Next they need to understand what that script<br>
> consumes, config.json [3]. The only reference for config.json is the files<br>
> that live in kolla-ansible, a mass of jinja and assumptions about how the<br>
> service will be run. Next, they need to figure out how to bind mount the<br>
> config files and config.json into the container in a way that can be<br>
> consumed by set_configs.py (which by the way, requires the base kolla image<br>
> in all cases). This is only for the config. For the service start up<br>
> command, this need to also be provided in config.json. This command is then<br>
> parsed out and written to a location in the image, which is consumed by a<br>
> series of start/extend start shell scripts. Kolla is *unique* in this<br>
> regard, no other project in the container world is interfacing with images<br>
> in this way. Being a snowflake in this regard is not a good thing. I'm still<br>
> waiting to hear from a real world operator who would prefer to spend time<br>
> learning the above to doing:<br>
<br>
You're pointing a very real documentation issue. I've mentioned in the<br>
other kolla thread that I have a stub for the kolla API documentation.<br>
I'll push a patch for what I have and we can iterate on that.<br>
<br>
> docker run -v /etc/keystone:/etc/keystone keystone:latest --entrypoint<br>
> /usr/bin/keystone [args]<br>
><br>
> This is the Docker API, it's easy to understand and pretty much the standard<br>
> at this point.<br>
<br>
Sure, using the docker API works for simpler cases, not too<br>
surprisingly once you start doing more funky things with your<br>
containers you're quickly reach the docker API limitations. That's<br>
when the kolla API comes in handy.<br>
See for example this recent patch<br>
<a href="https://review.openstack.org/#/c/556673/" rel="noreferrer noreferrer" target="_blank">https://review.openstack.org/#/c/556673/</a> where we needed to change<br>
some file permission to the uid/gid of the user inside the container.<br>
<br>
The first iteration basically used the docker API and started an<br>
additional container to fix the permissions:<br>
<br>
docker run -v<br>
/etc/pki/tls/certs/neutron.crt:/etc/pki/tls/certs/neutron.crt:rw \<br>
-v /etc/pki/tls/private/neutron.key:/etc/pki/tls/private/neutron.key:rw<br>
\<br>
neutron_image \<br>
/bin/bash -c 'chown neutron:neutron<br>
/etc/pki/tls/certs/neutron.crt; chown neutron:neutron<br>
/etc/pki/tls/private/neutron.key'<br>
<br>
You'll agree this is not the most obvious. And it had a nasty side<br>
effect that is changes the permissions of the files _on the host_.<br>
While using kolla API we could simply add to our config.json:<br>
<br>
- path: /etc/pki/tls/certs/neutron.crt<br>
owner: neutron:neutron<br>
- path: /etc/pki/tls/private/neutron.key<br>
owner: neutron:neutron<br>
<br>
> The other argument is that this removes the possibility for immutable<br>
> infrastructure. The concern is, with the new approach, a rookie operator<br>
> will modify one of the start scripts - resulting in uncertainty that what<br>
> was first deployed matches what is currently running. But with the way Kolla<br>
> is now, an operator can still do this! They can restart containers with a<br>
> custom entrypoint or additional bind mounts, they can exec in and change<br>
> config files, etc. etc. Kolla containers have never been immutable and we're<br>
> bending over backwards to artificially try and make this the case. We cant<br>
> protect a bad or inexperienced operator from shooting themselves in the<br>
> foot, there are better ways of doing so. If/when Docker or the upstream<br>
> container world solves this problem, it would then make sense for Kolla to<br>
> follow suit.<br>
><br>
> On the face of it, what the spec proposes is a simple change, it should not<br>
> radically pull the carpet out under people, or even change the way<br>
> kolla-ansible works in the near term. If consumers such as tripleo or other<br>
> parties feel it would in fact do so please do let me know and we can discuss<br>
> and mitigate these problems.<br>
<br>
TripleO uses these scripts extensively, we certainly do not want to<br>
see them go away from kolla images.<br>
<br>
Martin<br>
<br>
> Cheers,<br>
> -Paul<br>
><br>
> [0] <a href="https://review.openstack.org/#/c/550958/" rel="noreferrer noreferrer" target="_blank">https://review.openstack.org/#/c/550958/</a><br>
> [1] <a href="https://github.com/openstack/loci" rel="noreferrer noreferrer" target="_blank">https://github.com/openstack/loci</a><br>
> [2]<br>
> <a href="https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py" rel="noreferrer noreferrer" target="_blank">https://github.com/openstack/kolla/blob/master/docker/base/set_configs.py</a><br>
> [3]<br>
> <a href="https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/templates/keystone.json.j2" rel="noreferrer noreferrer" target="_blank">https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/templates/keystone.json.j2</a><br>
> [4]<br>
> <a href="http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-04-04-16.00.log.txt" rel="noreferrer noreferrer" target="_blank">http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-04-04-16.00.log.txt</a><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div></div></div>