Open Stack

Hi Zane, please count me in :-)


Cheers,
Lingxian Kong

On Wed, Jun 6, 2018 at 4:52 AM, Remo Mattei <remo at rm.ht> wrote:

> I will be happy to check it out.
>
> Remo
>
>
> On Jun 5, 2018, at 9:19 AM, Zane Bitter <zbitter at redhat.com> wrote:
>
> I've been doing some investigation into the Service Catalog in Kubernetes
> and how we can get OpenStack resources to show up in the catalog for use by
> applications running in Kubernetes. (The Big 3 public clouds already
> support this.) The short answer is via an implementation of something
> called the Open Service Broker API, but there are shortcuts available to
> make it easier to do.
>
> I'm convinced that this is readily achievable and something we ought to do
> as a community.
>
> I've put together a (long-winded) FAQ below to answer all of your
> questions about it.
>
> Would you be interested in working on a new project to implement this
> integration? Reply to this thread and let's collect a list of volunteers to
> form the initial core review team.
>
> cheers,
> Zane.
>
>
> What is the Open Service Broker API?
> ------------------------------------
>
> The Open Service Broker API[1] is a standard way to expose external
> resources to applications running in a PaaS. It was originally developed in
> the context of CloudFoundry, but the same standard was adopted by
> Kubernetes (and hence OpenShift) in the form of the Service Catalog
> extension[2]. (The Service Catalog in Kubernetes is the component that
> calls out to a service broker.) So a single implementation can cover the
> most popular open-source PaaS offerings.
>
> In many cases, the services take the form of simply a pre-packaged
> application that also runs inside the PaaS. But they don't have to be -
> services can be anything. Provisioning via the service broker ensures that
> the services requested are tied in to the PaaS's orchestration of the
> application's lifecycle.
>
> (This is certainly not the be-all and end-all of integration between
> OpenStack and containers - we also need ways to tie PaaS-based applications
> into the OpenStack's orchestration of a larger group of resources. Some
> applications may even use both. But it's an important part of the story.)
>
> What sorts of services would OpenStack expose?
> ----------------------------------------------
>
> Some example use cases might be:
>
> * The application needs a reliable message queue. Rather than spinning up
> multiple storage-backed containers with anti-affinity policies and dealing
> with the overhead of managing e.g. RabbitMQ, the application requests a
> Zaqar queue from an OpenStack cloud. The overhead of running the queueing
> service is amortised across all of the applications in the cloud. The queue
> gets cleaned up correctly when the application is removed, since it is tied
> into the application definition.
>
> * The application needs a database. Rather than spinning one up in a
> storage-backed container and dealing with the overhead of managing it, the
> application requests a Trove DB from an OpenStack cloud.
>
> * The application includes a service that needs to run on bare metal for
> performance reasons (e.g. could also be a database). The application
> requests a bare-metal server from Nova w/ Ironic for the purpose. (The same
> applies to requesting a VM, but there are alternatives like KubeVirt -
> which also operates through the Service Catalog - available for getting a
> VM in Kubernetes. There are no non-proprietary alternatives for getting a
> bare-metal server.)
>
> AWS[3], Azure[4], and GCP[5] all have service brokers available that
> support these and many more services that they provide. I don't know of any
> reason in principle not to expose every type of resource that OpenStack
> provides via a service broker.
>
> How is this different from cloud-provider-openstack?
> ----------------------------------------------------
>
> The Cloud Controller[6] interface in Kubernetes allows Kubernetes itself
> to access features of the cloud to provide its service. For example, if k8s
> needs persistent storage for a container then it can request that from
> Cinder through cloud-provider-openstack[7]. It can also request a load
> balancer from Octavia instead of having to start a container running
> HAProxy to load balance between multiple instances of an application
> container (thus enabling use of hardware load balancers via the cloud's
> abstraction for them).
>
> In contrast, the Service Catalog interface allows the *application*
> running on Kubernetes to access features of the cloud.
>
> What does a service broker look like?
> -------------------------------------
>
> A service broker provides an HTTP API with 5 actions:
>
> * List the services provided by the broker
> * Create an instance of a resource
> * Bind the resource into an instance of the application
> * Unbind the resource from an instance of the application
> * Delete the resource
>
> The binding step is used for things like providing a set of DB credentials
> to a container. You can rotate credentials when replacing a container by
> revoking the existing credentials on unbind and creating a new set on bind,
> without replacing the entire resource.
>
> Is there an easier way?
> -----------------------
>
> Yes! Folks from OpenShift came up with a project called the Automation
> Broker[8]. To add support for a service to Automation Broker you just
> create a container with an Ansible playbook to handle each of the actions
> (create/bind/unbind/delete). This eliminates the need to write another
> implementation of the service broker API, and allows us to simply write
> Ansible playbooks instead.[9]
>
> (Aside: Heat uses a comparable method to allow users to manage an external
> resource using Mistral workflows: the OS::Mistral::ExternalResource
> resource type.)
>
> Support for accessing AWS resources through a service broker is also
> implemented using these Ansible Playbook Bundles.[3]
>
> Does this mean maintaining another client interface?
> ----------------------------------------------------
>
> Maybe not. We already have per-project Python libraries, (deprecated)
> per-project CLIs, openstackclient CLIs, openstack-sdk, shade, Heat resource
> plugins, and Horizon dashboards. (Mistral actions are generated
> automatically from the clients.) Some consolidation is already planned, but
> it would be great not to require projects to maintain yet another interface.
>
> One option is to implement a tool that generates a set of playbooks for
> each of the resources already exposed (via shade) in the OpenStack Ansible
> modules. Then in theory we'd only need to implement the common parts once,
> and then every service with support in shade would get this for free.
> Ideally the same broker could be used against any OpenStack cloud (so e.g.
> k8s might be running in your private cloud, but you may want its service
> catalog to allow you to connect to resources in one or more public clouds)
> - using shade is an advantage there because it is designed to abstract the
> differences between clouds.
>
> Another option might be to write or generate Heat templates for each
> resource type we want to expose. Then we'd only need to implement a common
> way of creating a Heat stack, and just have a different template for each
> resource type. This is the approach taken by the AWS playbook bundles
> (except with CloudFormation, obviously). An advantage is that this allows
> Heat to do any checking and type conversion required on the input
> parameters. Heat templates can also be made to be fairly cloud-independent,
> mainly because they make it easier to be explicit about things like ports
> and subnets than on the command line, where it's more tempting to allow
> things to happen in a magical but cloud-specific way.
>
> I'd prefer to go with the pure-Ansible autogenerated way so we can have
> support for everything, but looking at the GCP[5]/Azure[4]/AWS[3] brokers
> they have 10, 11 and 17 services respectively, so arguably we could get a
> comparable number of features exposed without investing crazy amounts of
> time if we had to write templates explicitly.
>
> How would authentication work?
> ------------------------------
>
> There are two main deployment topologies we need to consider: Kubernetes
> deployed by an OpenStack tenant (Magnum-style, though not necessarily using
> Magnum) and accessing resources in that tenant's project in the local
> cloud, or accessing resources in some remote OpenStack cloud.
>
> We also need to take into account that in the second case, the Kubernetes
> cluster may 'belong' to a single cloud tenant (as in the first case) or may
> be shared by applications that each need to authenticate to different
> OpenStack tenants. (Kubernetes has traditionally assumed the former, but I
> expect it to move in the direction of allowing the latter, and it's already
> fairly common for OpenShift deployments.)
>
> The way e.g. the AWS broker[3] works is that you can either use the
> credentials provisioned to the VM that k8s is installed on (a 'Role' in AWS
> parlance - note that this is completely different to a Keystone Role), or
> supply credentials to authenticate to AWS remotely.
>
> OpenStack doesn't yet support per-instance credentials, although we're
> working on it. (One thing to keep in mind is that ideally we'll want a way
> to provide different permissions to the service broker and
> cloud-provider-openstack.) An option in the meantime might be to provide a
> way to set up credentials as part of the k8s installation. We'd also need
> to have a way to specify credentials manually. Unlike for proprietary
> clouds, the credentials also need to include the Keystone auth_url. We
> should try to reuse openstacksdk's clouds.yaml/secure.yaml format[10] if
> possible.
>
> The OpenShift Ansible Broker works by starting up an Ansible container on
> k8s to run a playbook from the bundle, so presumably credentials can be
> passed as regular k8s secrets.
>
> In all cases we'll want to encourage users to authenticate using Keystone
> Application Credentials[11].
>
> How would network integration work?
> -----------------------------------
>
> Kuryr[12] allows us to connect application containers in Kubernetes to
> Neutron networks in OpenStack. It would be desirable if, when the user
> requests a VM or bare-metal server through the service broker, it were
> possible to choose between attaching to the same network as Kubernetes
> pods, or to a different network.
>
>
> [1] https://www.openservicebrokerapi.org/
> [2] https://kubernetes.io/docs/concepts/service-catalog/
> [3] https://github.com/awslabs/aws-servicebroker#aws-service-broker
> [4] https://github.com/Azure/open-service-broker-azure#
> open-service-broker-for-azure
> [5] https://github.com/GoogleCloudPlatform/gcp-
> service-broker#cloud-foundry-service-broker-for-google-cloud-platform
> [6] https://github.com/kubernetes/community/blob/
> master/keps/0002-controller-manager.md#remove-cloud-
> provider-code-from-kubernetes-core
> [7] https://github.com/kubernetes/cloud-provider-
> openstack#openstack-cloud-controller-manager
> [8] http://automationbroker.io/
> [9] https://docs.openshift.org/latest/apb_devel/index.html
> [10] https://docs.openstack.org/openstacksdk/latest/user/
> config/configuration.html#config-files
> [11] https://docs.openstack.org/keystone/latest/user/
> application_credentials.html
> [12] https://docs.openstack.org/kuryr/latest/devref/goals_
> and_use_cases.html
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180606/2eed2f19/attachment-0001.html>