Open Stack

One benefit of the kolla API that I've not seen mentioned yet (sorry if I
missed it) is that you can change files on the host without affecting the
running container. Bind mounts don't have this property. This is handy for
reconfiguration/upgrade operations, where we write out a new set of config
before recreating/restarting the container. COPY_ONCE is the king of
immutable here, but even for COPY_ALWAYS, this works as long as the
container doesn't restart while the config files are being written.

Mark

On 5 April 2018 at 21:41, Michał Jastrzębski <inc007 at gmail.com> wrote:

> So I'll re-iterate comment which I made in BCN. In previous thread we
> praised how Kolla provided stable API for images, and I agree that it
> was great design choice (to provide stable API, not necessarily how
> API looks), and this change would break it. So *if* we decide to do
> it, we need to follow deprecation, that means we could deprecate these
> files in this release and start removing them in next.
>
> Support for LOCI in kolla-ansible is good thing, but I don't think
> changing Kolla image API is required for that. LOCI provides base
> image arument, so we could simply create base-image with all the
> extended-start and set-config mechanisms and some shim to source
> extended-start script that belongs to particular container. We will
> need kolla layer image anyway because set_config is there to stay (as
> Martin pointed out it's valuable tool fixing real issue and it's used
> by more projects than just kolla-ansible). We could add another script
> that would look like extended_start.sh -> source
> $CONTAINER_NAME-extended-start.sh and copy all kolla's extended start
> scripts to dir with proper naming (I believe this is solution that Sam
> came up with shortly after BCN). This is purely techincal and not that
> hard to do, much quicker and easier than deprecating API...
>
> On 5 April 2018 at 12:28, Martin André <m.andre at redhat.com> wrote:
> > On Thu, Apr 5, 2018 at 2:16 PM, Paul Bourke <paul.bourke at oracle.com>
> wrote:
> >> Hi all,
> >>
> >> This mail is to serve as a follow on to the discussion during
> yesterday's
> >> team meeting[4], which was regarding the desire to move start scripts
> out of
> >> the kolla images [0]. There's a few factors at play, and it may well be
> best
> >> left to discuss in person at the summit in May, but hopefully we can
> get at
> >> least some of this hashed out before then.
> >>
> >> I'll start by summarising why I think this is a good idea, and then
> attempt
> >> to address some of the concerns that have come up since.
> >>
> >> First off, to be frank, this is effort is driven by wanting to add
> support
> >> for loci images[1] in kolla-ansible. I think it would be unreasonable
> for
> >> anyone to argue this is a bad objective to have, loci images have very
> >> obvious benefits over what we have in Kolla today. I'm not looking to
> drop
> >> support for Kolla images at all, I simply want to continue decoupling
> things
> >> to the point where operators can pick and choose what works best for
> them.
> >> Stemming from this, I think moving these scripts out of the images
> provides
> >> a clear benefit to our consumers, both users of kolla and third parties
> such
> >> as triple-o. Let me explain why.
> >
> > It's still very obscure to me how removing the scripts from kolla
> > images will benefit consumers. If the reason is that you want to
> > re-use them in other, non-kolla images, I believe we should package
> > the scripts. I've left some comments in your spec review.
> >
> >> Normally, to run a docker image, a user will do 'docker run
> >> helloworld:latest'. In any non trivial application, config needs to be
> >> provided. In the vast majority of cases this is either provided via a
> bind
> >> mount (docker run -v hello.conf:/etc/hello.conf helloworld:latest), or
> via
> >> environment variables (docker run --env HELLO=paul helloworld:latest).
> This
> >> is all bog standard stuff, something anyone who's spent an hour learning
> >> docker can understand.
> >>
> >> Now, lets say someone wants to try out OpenStack with Docker, and they
> look
> >> at Kolla. First off they have to look at something called
> set_configs.py[2]
> >> - over 400 lines of Python. Next they need to understand what that
> script
> >> consumes, config.json [3]. The only reference for config.json is the
> files
> >> that live in kolla-ansible, a mass of jinja and assumptions about how
> the
> >> service will be run. Next, they need to figure out how to bind mount the
> >> config files and config.json into the container in a way that can be
> >> consumed by set_configs.py (which by the way, requires the base kolla
> image
> >> in all cases). This is only for the config. For the service start up
> >> command, this need to also be provided in config.json. This command is
> then
> >> parsed out and written to a location in the image, which is consumed by
> a
> >> series of start/extend start shell scripts. Kolla is *unique* in this
> >> regard, no other project in the container world is interfacing with
> images
> >> in this way. Being a snowflake in this regard is not a good thing. I'm
> still
> >> waiting to hear from a real world operator who would prefer to spend
> time
> >> learning the above to doing:
> >
> > You're pointing a very real documentation issue. I've mentioned in the
> > other kolla thread that I have a stub for the kolla API documentation.
> > I'll push a patch for what I have and we can iterate on that.
> >
> >>   docker run -v /etc/keystone:/etc/keystone keystone:latest --entrypoint
> >> /usr/bin/keystone [args]
> >>
> >> This is the Docker API, it's easy to understand and pretty much the
> standard
> >> at this point.
> >
> > Sure, using the docker API works for simpler cases, not too
> > surprisingly once you start doing more funky things with your
> > containers you're quickly reach the docker API limitations. That's
> > when the kolla API comes in handy.
> > See for example this recent patch
> > https://review.openstack.org/#/c/556673/ where we needed to change
> > some file permission to the uid/gid of the user inside the container.
> >
> > The first iteration basically used the docker API and started an
> > additional container to fix the permissions:
> >
> >   docker run -v
> > /etc/pki/tls/certs/neutron.crt:/etc/pki/tls/certs/neutron.crt:rw \
> >         -v /etc/pki/tls/private/neutron.key:/etc/pki/tls/private/
> neutron.key:rw
> > \
> >         neutron_image \
> >         /bin/bash -c 'chown neutron:neutron
> > /etc/pki/tls/certs/neutron.crt; chown neutron:neutron
> > /etc/pki/tls/private/neutron.key'
> >
> > You'll agree this is not the most obvious. And it had a nasty side
> > effect that is changes the permissions of the files _on the host_.
> > While using kolla API we could simply add to our config.json:
> >
> >   - path: /etc/pki/tls/certs/neutron.crt
> >     owner: neutron:neutron
> >   - path: /etc/pki/tls/private/neutron.key
> >     owner: neutron:neutron
> >
> >> The other argument is that this removes the possibility for immutable
> >> infrastructure. The concern is, with the new approach, a rookie operator
> >> will modify one of the start scripts - resulting in uncertainty that
> what
> >> was first deployed matches what is currently running. But with the way
> Kolla
> >> is now, an operator can still do this! They can restart containers with
> a
> >> custom entrypoint or additional bind mounts, they can exec in and change
> >> config files, etc. etc. Kolla containers have never been immutable and
> we're
> >> bending over backwards to artificially try and make this the case. We
> cant
> >> protect a bad or inexperienced operator from shooting themselves in the
> >> foot, there are better ways of doing so. If/when Docker or the upstream
> >> container world solves this problem, it would then make sense for Kolla
> to
> >> follow suit.
> >>
> >> On the face of it, what the spec proposes is a simple change, it should
> not
> >> radically pull the carpet out under people, or even change the way
> >> kolla-ansible works in the near term. If consumers such as tripleo or
> other
> >> parties feel it would in fact do so please do let me know and we can
> discuss
> >> and mitigate these problems.
> >
> > TripleO uses these scripts extensively, we certainly do not want to
> > see them go away from kolla images.
> >
> > Martin
> >
> >> Cheers,
> >> -Paul
> >>
> >> [0] https://review.openstack.org/#/c/550958/
> >> [1] https://github.com/openstack/loci
> >> [2]
> >> https://github.com/openstack/kolla/blob/master/docker/base/
> set_configs.py
> >> [3]
> >> https://github.com/openstack/kolla-ansible/blob/master/
> ansible/roles/keystone/templates/keystone.json.j2
> >> [4]
> >> http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.
> 2018-04-04-16.00.log.txt
> >>
> >> ____________________________________________________________
> ______________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > ____________________________________________________________
> ______________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180406/68c28fb3/attachment.html>