[openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints

Florian Engelmann florian.engelmann at everyware.ch
Tue Oct 9 11:02:20 UTC 2018

Am 10/9/18 um 11:04 AM schrieb Mark Goddard:
> Thanks for these suggestions Florian, there are some interesting ideas 
> in here. I'm a little concerned about the maintenance overhead of adding 
> support for all of these things, and wonder if some of them could be 
> done without explicit support in kolla and kolla-ansible. The kolla 
> projects have been able to move quickly by providing a flexible 
> configuration mechanism that avoids the need to maintain support for 
> every OpenStack feature. Other thoughts inline.

I do understand your apprehensions Mark. For some of the suggested 
changes/additions I agree. But adding those components without 
kolla/kolla-ansible integration feels not right.

> On Mon, 8 Oct 2018 at 11:15, Florian Engelmann 
> <florian.engelmann at everyware.ch <mailto:florian.engelmann at everyware.ch>> 
> wrote:
>     Hi,
>     I would like to start a discussion about some changes and additions I
>     would like to see in in kolla and kolla-ansible.
>     1. Keepalived is a problem in layer3 spine leaf networks as any
>     floating
>     IP can only exist in one leaf (and VRRP is a problem in layer3). I
>     would
>     like to use consul and registrar to get rid of the "internal" floating
>     IP and use consuls DNS service discovery to connect all services with
>     each other.
> Without reading up, I'm not sure exactly how this fits together. If 
> kolla-ansible made the API host configurable for each service rather 
> than globally, would that be a step in the right direction?

No that would not help. The problem is HA. Right now there is a 
"central" floating IP (kolla_internal_vip_address) that is used for all 
services to connect to (each other). Keepalived is failing that IP over 
if the "active" host fails. In a layer3 (CLOS/Spine-Leaf) network this 
IP is only available in one leaf/rack. So that rack is a "SPOF".
Using service discovery fits perfect in a CLOS network and scales much 
better as a HA solution.

>     2. Using "ports" for external API (endpoint) access is a major headache
>     if a firewall is involved. I would like to configure the HAProxy (or
>     fabio) for the external access to use "Host:" like, eg. "Host:
>     keystone.somedomain.tld", "Host: nova.somedomain.tld", ... with HTTPS.
>     Any customer would just need HTTPS access and not have to open all
>     those
>     ports in his firewall. For some enterprise customers it is not possible
>     to request FW changes like that.
>     3. HAProxy is not capable to handle "read/write" split with Galera. I
>     would like to introduce ProxySQL to be able to scale Galera.
> It's now possible to use an external database server with kolla-ansible, 
> instead of deploying a mariadb/galera cluster. This could be implemented 
> how you like, see 
> https://docs.openstack.org/kolla-ansible/latest/reference/external-mariadb-guide.html.

Yes I agree. And this is what we will do in our first production 
deployment. But I would love to see ProxySQL in Kolla as well.
As a side note: Kolla-ansible does use:

option mysql-check user haproxy post-41

to check Galera, but that check does not fail if the node is out of sync 
with the other nodes!


>     4. HAProxy is fine but fabio integrates well with consul, statsd and
>     could be connected to a vault cluster to manage secure certificate
>     access.
> As above.
>     5. I would like to add vault as Barbican backend.
> Does this need explicit support in kolla and kolla-ansible, or could it 
> be done through configuration of barbican.conf? Are there additional 
> packages required in the barbican container? If so, see 
> https://docs.openstack.org/kolla/latest/admin/image-building.html#package-customisation.

True but the vault (and consul) containers could be deployed and managed 
by kolla-ansible.

>     6. I would like to add an option to enable tokenless authentication for
>     all services with each other to get rid of all the openstack service
>     passwords (security issue).
> Again, could this be done without explicit support?

We did not investigate here. Changes to the apache configuration are 
needed. I guess we will have to change the kolla container itself to do 
so? Is it possible to "inject" files in a container using kolla-ansible?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5210 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20181009/35b849b0/attachment.bin>

More information about the OpenStack-dev mailing list