Open Stack

On 03.04.2017 21:01, Dan Prince wrote:
> On Mon, 2017-04-03 at 16:15 +0200, Bogdan Dobrelya wrote:
>> Let's please re-evaluate configuration of containerized non-
>> openstack,
>> like database, message queue, key value, web, services for tripleo
>> heat
>> templates and Kolla. Here is an example containerized etcd patch [0].
>>
>> tl;dr use kolla images and bootsrap OR upstream images with direct
>> commands:
>>
>> .. code-block:: yaml
>> kolla_config:
>>     /var/lib/kolla/config_files/foo.json
>>       command: /usr/bin/foo
>>
>> vs
>>
>> .. code-block:: yaml
>> foo:
>>     image: upstream/foo:latest
>>     command: /usr/bin/foo
>>
>> Note, tht already doesn't use configs [1] copied into the images by
>> kolla build time. The next and logical step might be to omit kolla's
>> bootstrap, where applicable, as well.
> 
> The kolla config file copying proved to be a bit pedantic. So we
> removed it. A good example of this would be how this played out for the
> keystone service:
> 
> http://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?
> id=332e8ec10345ad5c8bf10a532f6f6003da682b68
> 
>>
>> There is a two options:
>> * use kolla images and bootstrap as t-h-t does now for all services
>> being containerized
> 
> We do not use KOLLA_BOOTSTRAP for all services now. Only 4 services use
> it currently I think. I think the general consensus is we should not be
> using it unless there is a functional requirement to do so. Services
> like Mysql and Rabbitmq have some extra initialization that needs to
> execute in-container before the services startup. We could duplicate
> this in tripleo-heat-templates, run it with docker-cmd perhaps and do
> it that way but we initially made exception for just a few services.
> 
> Other services like Glance are using it, but that is just historical.
> There is already a patch to remove the use of KOLLA_BOOTSTRAP for this
> service:
> 
> https://review.openstack.org/#/c/440884/1
> 
> So in summary the consensus is we'd prefer not to be using the
> KOLLA_BOOTSTRAP environment variables because in some cases there is a
> 'Kolla' flavor to these things that don't match how TripleO deploys
> things.
> 
> It is worth pointing out that while we aren't using KOLLA_BOOTSTRAP we
> are using the kolla startup systems in many cases. This gives some
> features around file permissions, extra sudoers files, etc. We may be
> able to stop using this for some services but I also think we are
> getting value out of the interfaces today. They aren't nearly as
> verbose as the Kolla config copying stuff so we could go either way.

That's interesting, there is related topic with hostpath mounted
persistent log dirs [0]. The issue is:

[tl;dr] kolla_config is a bad fit for logs, let's run containers' steps
as 'user: root' in a user namespace remapped [1] for a tripleocontainers
system user. Thoughts? And would that work for stock centos7/rhel7 kernels?


So, the issue is when log/data dirs created by host_prep_tasks, we must
configure host permissions in a simple non-opinionated way.
What I can see for t-h-t to cover it on its own:
* Custom entrypoints for containers adds complexity and head ache.
* Mount logs world writable is a non starter.
* Running containers as a user root is a security issue.

What I can see for Kolla to accomplish that task:
* the extended start does not cover logs and there are nuances with
recursive chowning and permissions of new files [2].
* log directories doesn't fit well into the kolla_config's volumes and
permissions defined in services' config.json. Using those for log
volumes would look like a dirty hack. This also would undo the recent
changes with using hostpath mounts instead of the kolla_config volumes.

That's why I came to the option with 'user: root' and remapped user
namespace for docker daemon. Please share your thoughts.

[0] https://review.openstack.org/#/c/442603/
[1] https://goo.gl/HtDQYk
[2] https://review.openstack.org/#/c/452742/

> 
>> pros: same way to template everything, kolla build/start just works.
>> risks: non-openstack services, eventually, may stop being supported
>> by
>> Kolla for number of reasons. Kolla bootstrap changes aren't tested in
>> tripleo CI and might be breaking
>> cons: locking in to the kolla opinionated bootstrap entry points and
>> kolla_config's config.json and command.
>>
>> * if applicable to the service, use upstream images (etcd example
>> [2])
>> w/o any kolla parts.
>> pros: less moving parts like custom entry points, no locking in
>> onto opinionated Kolla config/bootstrap
>> risks: Upstream image changes aren't tested in tripleo CI and might
>> be
>> breaking
>> cons: different ways to template openstack/non-openstack services,
>> kolla
>> build/start doesn't work for the latter.
>>
>> [0] https://review.openstack.org/#/c/447627/
>> [1] https://review.openstack.org/#/c/451366
>> [2]
>> https://review.openstack.org/#/c/445883/2/contrib/overcloud_container
>> s.yaml
>>
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando