Open Stack

I will be happy to check it out.

Remo 

> On Jun 5, 2018, at 9:19 AM, Zane Bitter <zbitter at redhat.com> wrote:
> 
> I've been doing some investigation into the Service Catalog in Kubernetes and how we can get OpenStack resources to show up in the catalog for use by applications running in Kubernetes. (The Big 3 public clouds already support this.) The short answer is via an implementation of something called the Open Service Broker API, but there are shortcuts available to make it easier to do.
> 
> I'm convinced that this is readily achievable and something we ought to do as a community.
> 
> I've put together a (long-winded) FAQ below to answer all of your questions about it.
> 
> Would you be interested in working on a new project to implement this integration? Reply to this thread and let's collect a list of volunteers to form the initial core review team.
> 
> cheers,
> Zane.
> 
> 
> What is the Open Service Broker API?
> ------------------------------------
> 
> The Open Service Broker API[1] is a standard way to expose external resources to applications running in a PaaS. It was originally developed in the context of CloudFoundry, but the same standard was adopted by Kubernetes (and hence OpenShift) in the form of the Service Catalog extension[2]. (The Service Catalog in Kubernetes is the component that calls out to a service broker.) So a single implementation can cover the most popular open-source PaaS offerings.
> 
> In many cases, the services take the form of simply a pre-packaged application that also runs inside the PaaS. But they don't have to be - services can be anything. Provisioning via the service broker ensures that the services requested are tied in to the PaaS's orchestration of the application's lifecycle.
> 
> (This is certainly not the be-all and end-all of integration between OpenStack and containers - we also need ways to tie PaaS-based applications into the OpenStack's orchestration of a larger group of resources. Some applications may even use both. But it's an important part of the story.)
> 
> What sorts of services would OpenStack expose?
> ----------------------------------------------
> 
> Some example use cases might be:
> 
> * The application needs a reliable message queue. Rather than spinning up multiple storage-backed containers with anti-affinity policies and dealing with the overhead of managing e.g. RabbitMQ, the application requests a Zaqar queue from an OpenStack cloud. The overhead of running the queueing service is amortised across all of the applications in the cloud. The queue gets cleaned up correctly when the application is removed, since it is tied into the application definition.
> 
> * The application needs a database. Rather than spinning one up in a storage-backed container and dealing with the overhead of managing it, the application requests a Trove DB from an OpenStack cloud.
> 
> * The application includes a service that needs to run on bare metal for performance reasons (e.g. could also be a database). The application requests a bare-metal server from Nova w/ Ironic for the purpose. (The same applies to requesting a VM, but there are alternatives like KubeVirt - which also operates through the Service Catalog - available for getting a VM in Kubernetes. There are no non-proprietary alternatives for getting a bare-metal server.)
> 
> AWS[3], Azure[4], and GCP[5] all have service brokers available that support these and many more services that they provide. I don't know of any reason in principle not to expose every type of resource that OpenStack provides via a service broker.
> 
> How is this different from cloud-provider-openstack?
> ----------------------------------------------------
> 
> The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself to access features of the cloud to provide its service. For example, if k8s needs persistent storage for a container then it can request that from Cinder through cloud-provider-openstack[7]. It can also request a load balancer from Octavia instead of having to start a container running HAProxy to load balance between multiple instances of an application container (thus enabling use of hardware load balancers via the cloud's abstraction for them).
> 
> In contrast, the Service Catalog interface allows the *application* running on Kubernetes to access features of the cloud.
> 
> What does a service broker look like?
> -------------------------------------
> 
> A service broker provides an HTTP API with 5 actions:
> 
> * List the services provided by the broker
> * Create an instance of a resource
> * Bind the resource into an instance of the application
> * Unbind the resource from an instance of the application
> * Delete the resource
> 
> The binding step is used for things like providing a set of DB credentials to a container. You can rotate credentials when replacing a container by revoking the existing credentials on unbind and creating a new set on bind, without replacing the entire resource.
> 
> Is there an easier way?
> -----------------------
> 
> Yes! Folks from OpenShift came up with a project called the Automation Broker[8]. To add support for a service to Automation Broker you just create a container with an Ansible playbook to handle each of the actions (create/bind/unbind/delete). This eliminates the need to write another implementation of the service broker API, and allows us to simply write Ansible playbooks instead.[9]
> 
> (Aside: Heat uses a comparable method to allow users to manage an external resource using Mistral workflows: the OS::Mistral::ExternalResource resource type.)
> 
> Support for accessing AWS resources through a service broker is also implemented using these Ansible Playbook Bundles.[3]
> 
> Does this mean maintaining another client interface?
> ----------------------------------------------------
> 
> Maybe not. We already have per-project Python libraries, (deprecated) per-project CLIs, openstackclient CLIs, openstack-sdk, shade, Heat resource plugins, and Horizon dashboards. (Mistral actions are generated automatically from the clients.) Some consolidation is already planned, but it would be great not to require projects to maintain yet another interface.
> 
> One option is to implement a tool that generates a set of playbooks for each of the resources already exposed (via shade) in the OpenStack Ansible modules. Then in theory we'd only need to implement the common parts once, and then every service with support in shade would get this for free. Ideally the same broker could be used against any OpenStack cloud (so e.g. k8s might be running in your private cloud, but you may want its service catalog to allow you to connect to resources in one or more public clouds) - using shade is an advantage there because it is designed to abstract the differences between clouds.
> 
> Another option might be to write or generate Heat templates for each resource type we want to expose. Then we'd only need to implement a common way of creating a Heat stack, and just have a different template for each resource type. This is the approach taken by the AWS playbook bundles (except with CloudFormation, obviously). An advantage is that this allows Heat to do any checking and type conversion required on the input parameters. Heat templates can also be made to be fairly cloud-independent, mainly because they make it easier to be explicit about things like ports and subnets than on the command line, where it's more tempting to allow things to happen in a magical but cloud-specific way.
> 
> I'd prefer to go with the pure-Ansible autogenerated way so we can have support for everything, but looking at the GCP[5]/Azure[4]/AWS[3] brokers they have 10, 11 and 17 services respectively, so arguably we could get a comparable number of features exposed without investing crazy amounts of time if we had to write templates explicitly.
> 
> How would authentication work?
> ------------------------------
> 
> There are two main deployment topologies we need to consider: Kubernetes deployed by an OpenStack tenant (Magnum-style, though not necessarily using Magnum) and accessing resources in that tenant's project in the local cloud, or accessing resources in some remote OpenStack cloud.
> 
> We also need to take into account that in the second case, the Kubernetes cluster may 'belong' to a single cloud tenant (as in the first case) or may be shared by applications that each need to authenticate to different OpenStack tenants. (Kubernetes has traditionally assumed the former, but I expect it to move in the direction of allowing the latter, and it's already fairly common for OpenShift deployments.)
> 
> The way e.g. the AWS broker[3] works is that you can either use the credentials provisioned to the VM that k8s is installed on (a 'Role' in AWS parlance - note that this is completely different to a Keystone Role), or supply credentials to authenticate to AWS remotely.
> 
> OpenStack doesn't yet support per-instance credentials, although we're working on it. (One thing to keep in mind is that ideally we'll want a way to provide different permissions to the service broker and cloud-provider-openstack.) An option in the meantime might be to provide a way to set up credentials as part of the k8s installation. We'd also need to have a way to specify credentials manually. Unlike for proprietary clouds, the credentials also need to include the Keystone auth_url. We should try to reuse openstacksdk's clouds.yaml/secure.yaml format[10] if possible.
> 
> The OpenShift Ansible Broker works by starting up an Ansible container on k8s to run a playbook from the bundle, so presumably credentials can be passed as regular k8s secrets.
> 
> In all cases we'll want to encourage users to authenticate using Keystone Application Credentials[11].
> 
> How would network integration work?
> -----------------------------------
> 
> Kuryr[12] allows us to connect application containers in Kubernetes to Neutron networks in OpenStack. It would be desirable if, when the user requests a VM or bare-metal server through the service broker, it were possible to choose between attaching to the same network as Kubernetes pods, or to a different network.
> 
> 
> [1] https://www.openservicebrokerapi.org/ <https://www.openservicebrokerapi.org/>
> [2] https://kubernetes.io/docs/concepts/service-catalog/ <https://kubernetes.io/docs/concepts/service-catalog/>
> [3] https://github.com/awslabs/aws-servicebroker#aws-service-broker <https://github.com/awslabs/aws-servicebroker#aws-service-broker>
> [4] https://github.com/Azure/open-service-broker-azure#open-service-broker-for-azure <https://github.com/Azure/open-service-broker-azure#open-service-broker-for-azure>
> [5] https://github.com/GoogleCloudPlatform/gcp-service-broker#cloud-foundry-service-broker-for-google-cloud-platform <https://github.com/GoogleCloudPlatform/gcp-service-broker#cloud-foundry-service-broker-for-google-cloud-platform>
> [6] https://github.com/kubernetes/community/blob/master/keps/0002-controller-manager.md#remove-cloud-provider-code-from-kubernetes-core <https://github.com/kubernetes/community/blob/master/keps/0002-controller-manager.md#remove-cloud-provider-code-from-kubernetes-core>
> [7] https://github.com/kubernetes/cloud-provider-openstack#openstack-cloud-controller-manager <https://github.com/kubernetes/cloud-provider-openstack#openstack-cloud-controller-manager>
> [8] http://automationbroker.io/ <http://automationbroker.io/>
> [9] https://docs.openshift.org/latest/apb_devel/index.html <https://docs.openshift.org/latest/apb_devel/index.html>
> [10] https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html#config-files <https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html#config-files>
> [11] https://docs.openstack.org/keystone/latest/user/application_credentials.html <https://docs.openstack.org/keystone/latest/user/application_credentials.html>
> [12] https://docs.openstack.org/kuryr/latest/devref/goals_and_use_cases.html <https://docs.openstack.org/kuryr/latest/devref/goals_and_use_cases.html>
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org <mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180605/fc648c67/attachment.html>