[openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints

Mark Goddard mark at stackhpc.com
Tue Oct 9 11:47:07 UTC 2018


On Tue, 9 Oct 2018 at 12:03, Florian Engelmann <
florian.engelmann at everyware.ch> wrote:

> Am 10/9/18 um 11:04 AM schrieb Mark Goddard:
> > Thanks for these suggestions Florian, there are some interesting ideas
> > in here. I'm a little concerned about the maintenance overhead of adding
> > support for all of these things, and wonder if some of them could be
> > done without explicit support in kolla and kolla-ansible. The kolla
> > projects have been able to move quickly by providing a flexible
> > configuration mechanism that avoids the need to maintain support for
> > every OpenStack feature. Other thoughts inline.
> >
>
> I do understand your apprehensions Mark. For some of the suggested
> changes/additions I agree. But adding those components without
> kolla/kolla-ansible integration feels not right.
>

I'm not entirely against adding some of these things, if enough people in
the community want them. I'd just like to make sure that if they can be
done in a sane way without changes, then we do that and document how to do
it instead.

>
> >
> > On Mon, 8 Oct 2018 at 11:15, Florian Engelmann
> > <florian.engelmann at everyware.ch <mailto:florian.engelmann at everyware.ch>>
>
> > wrote:
> >
> >     Hi,
> >
> >     I would like to start a discussion about some changes and additions I
> >     would like to see in in kolla and kolla-ansible.
> >
> >     1. Keepalived is a problem in layer3 spine leaf networks as any
> >     floating
> >     IP can only exist in one leaf (and VRRP is a problem in layer3). I
> >     would
> >     like to use consul and registrar to get rid of the "internal"
> floating
> >     IP and use consuls DNS service discovery to connect all services with
> >     each other.
> >
> >
> > Without reading up, I'm not sure exactly how this fits together. If
> > kolla-ansible made the API host configurable for each service rather
> > than globally, would that be a step in the right direction?
>
> No that would not help. The problem is HA. Right now there is a
> "central" floating IP (kolla_internal_vip_address) that is used for all
> services to connect to (each other). Keepalived is failing that IP over
> if the "active" host fails. In a layer3 (CLOS/Spine-Leaf) network this
> IP is only available in one leaf/rack. So that rack is a "SPOF".
> Using service discovery fits perfect in a CLOS network and scales much
> better as a HA solution.
>
> Right, but what I'm saying as a thought experiment is, if we gave you the
required variables in kolla-ansible (e.g. nova_internal_fqdn) to make this
possible with an externally managed consul service, could that work?

>
> >
> >     2. Using "ports" for external API (endpoint) access is a major
> headache
> >     if a firewall is involved. I would like to configure the HAProxy (or
> >     fabio) for the external access to use "Host:" like, eg. "Host:
> >     keystone.somedomain.tld", "Host: nova.somedomain.tld", ... with
> HTTPS.
> >     Any customer would just need HTTPS access and not have to open all
> >     those
> >     ports in his firewall. For some enterprise customers it is not
> possible
> >     to request FW changes like that.
> >
> >     3. HAProxy is not capable to handle "read/write" split with Galera. I
> >     would like to introduce ProxySQL to be able to scale Galera.
> >
> >
> > It's now possible to use an external database server with kolla-ansible,
> > instead of deploying a mariadb/galera cluster. This could be implemented
> > how you like, see
> >
> https://docs.openstack.org/kolla-ansible/latest/reference/external-mariadb-guide.html
> .
>
> Yes I agree. And this is what we will do in our first production
> deployment. But I would love to see ProxySQL in Kolla as well.
> As a side note: Kolla-ansible does use:
>
> option mysql-check user haproxy post-41
>
> to check Galera, but that check does not fail if the node is out of sync
> with the other nodes!
>
> http://galeracluster.com/documentation-webpages/monitoringthecluster.html
>
> That's good to know. Could you raise a bug in kolla-ansible on launchpad,
and offer advice on how to improve this check if you have any?

>
> >
> >     4. HAProxy is fine but fabio integrates well with consul, statsd and
> >     could be connected to a vault cluster to manage secure certificate
> >     access.
> >
> > As above.
> >
> >     5. I would like to add vault as Barbican backend.
> >
> > Does this need explicit support in kolla and kolla-ansible, or could it
> > be done through configuration of barbican.conf? Are there additional
> > packages required in the barbican container? If so, see
> >
> https://docs.openstack.org/kolla/latest/admin/image-building.html#package-customisation
> .
>
> True but the vault (and consul) containers could be deployed and managed
> by kolla-ansible.
>
> I'd like to see if anyone else is interested in this. Kolla ansible
already deploys a large number of services, which is great. As with many
other projects I'm seeing the resources of core contributors fall off a
little, and I think we need to consider how to ensure the project is
maintainable long term. In my view a good way of doing that is to enable
integration with existing services, rather than deploying them. We need to
decide where the line is as a community. We have an IRC meeting at 3pm UTC
if you'd like to bring it up then.

>
> >     6. I would like to add an option to enable tokenless authentication
> for
> >     all services with each other to get rid of all the openstack service
> >     passwords (security issue).
> >
> > Again, could this be done without explicit support?
>
> We did not investigate here. Changes to the apache configuration are
> needed. I guess we will have to change the kolla container itself to do
> so? Is it possible to "inject" files in a container using kolla-ansible?
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20181009/739b3954/attachment.html>


More information about the OpenStack-dev mailing list