Open Stack

Team,

This thread is a continuation of a branch of the previous High Availability thread [1]. As the Magnum PTL, I’ve been aware of a number of different groups who have started using Magnum in recent months. For various reasons, there have been multiple requests for information about how to turn off the dependency on Barbican, which we use for secure storage of TLS certificates that are used to secure communications between various components of the software hosted on Magnum Bay resources. Examples of this are Docker Swarm, and Kubernetes, which we affectionately refer to as COEs (Container Orchestration Engines). The only alternative to Barbican currently offered in Magnum is a local file option, which is only intended to be used for testing, as the certificates are stored unencrypted on a local filesystem where the conductor runs, and when you use this option, you can’t scale beyond a single conductor.

Although our whole community agrees that using Barbican is the right long term solution for deployments of Magnum, we still wish to make the friction of adopting Magnum to be as low as possible without completely compromising all security best practices. Some ops teams are willing to adopt a new service, but not two. They only want to add Magnum and not Barbican. We think that once those operators become familiar with Magnum, adding Barbican will follow. In the mean time, we’d like to offer a Barbican alternative that allows Magnum to scale beyond one conductor, and allows for encrypted storage of TLC credentials needed for unattended bay operations. A blueprint [2] was recently proposed to address this. We discussed this in our team meeting today [3], where we used an etherpad [4] to collaborate on options that could be used as alternatives besides the ones offered today. This thread is not intended to answer how to make Barbican easier to adopt, but rather how to make Magnum easier to adopt while keeping Barbican as the default best-practice choice for certificate storage.

I want to highlight that the implementation of the spec referenced by Daneyon Hansen in his quoted response below was completed in the Liberty release timeframe, and communication between COE components is now secured using TLS. We are discussing the continued use of TLS for encrypted connections between COE components, but potentially using Keystone tokens for authentication between clients and COE’s rather than using TLS for both encryption and authentication. Further notes on this are available in the etherpad [4].

I ask that you please review the options under consideration, note your remarks in the etherpad [4], and continue discussion here as needed.

Thanks,

Adrian

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/089684.html
[2] https://blueprints.launchpad.net/magnum/+spec/barbican-alternative-store
[3] http://eavesdrop.openstack.org/meetings/containers/2016/containers.2016-03-22-16.01.html
[4] https://etherpad.openstack.org/p/magnum-barbican-alternative

On Mar 22, 2016, at 11:52 AM, Daneyon Hansen (danehans) <danehans at cisco.com<mailto:danehans at cisco.com>> wrote:



From: Hongbin Lu <hongbin.lu at huawei.com<mailto:hongbin.lu at huawei.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, March 21, 2016 at 8:19 PM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [magnum] High Availability

Tim,

Thanks for your advice. I respect your point of view and we will definitely encourage our users to try Barbican if they see fits. However, for the sake of Magnum, I think we have to decouple from Barbican at current stage. The coupling of Magnum and Barbican will increase the size of the system by two (1 project -> 2 project), which will significant increase the overall complexities.
·         For developers, it incurs significant overheads on development, quality assurance, and maintenance.
·         For operators, it doubles the amount of efforts of deploying and monitoring the system.
·         For users, a large system is likely to be unstable and fragile which affects the user experience.
In my point of view, I would like to minimize the system we are going to ship, so that we can reduce the overheads of maintenance and provides a stable system to our users.

I noticed that there are several suggestions to “force” our users to install Barbican, which I would respectfully disagree. Magnum is a young project and we are struggling to increase the adoption rate. I think we need to be nice to our users, otherwise, they will choose our competitors (there are container service everywhere). Please understand that we are not a mature project, like Nova, who has thousands of users. We really don’t have the power to force our users to do what they don’t like to do.

I also recognized there are several disagreements from the Barbican team. Per my understanding, most of the complaints are about the re-invention of Barbican equivalent functionality in Magnum. To address that, I am going to propose an idea to achieve the goal without duplicating Barbican. In particular, I suggest to add support for additional authentication system (Keystone in particular) for our Kubernetes bay (potentially for swarm/mesos). As a result, users can specify how to secure their bay’s API endpoint:
·         TLS: This option requires Barbican to be installed for storing the TLS certificates.
·         Keystone: This option doesn’t require Barbican. Users will use their OpenStack credentials to log into Kubernetes.

I believe this is a sensible option that addresses the original problem statement in [1]:

"Magnum currently controls Kubernetes API services using unauthenticated HTTP. If an attacker knows the api_address of a Kubernetes Bay, (s)he can control the cluster without any access control."

The [1] problem statement is authenticating the bay API endpoint, not encrypting it. With the option you propose, we can leave the existing tls-disabled attribute alone and continue supporting encryption. Using Keystone to authenticate the Kubernetes API already exists outside of Magnum in Hypernetes [2]. We will need to investigate support for the other coe types.

[1] https://github.com/openstack/magnum/blob/master/specs/tls-support-magnum.rst
[2] http://thenewstack.io/hypernetes-brings-multi-tenancy-microservices/



I am going to send another ML to describe the details. You are welcome to provide your inputs. Thanks.

Best regards,
Hongbin

From: Tim Bell [mailto:Tim.Bell at cern.ch]
Sent: March-19-16 5:55 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum] High Availability


From: Hongbin Lu <hongbin.lu at huawei.com<mailto:hongbin.lu at huawei.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Saturday 19 March 2016 at 04:52
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [magnum] High Availability

...
If you disagree, I would request you to justify why this approach works for Heat but not for Magnum. Also, I also wonder if Heat has a plan to set a hard dependency on Barbican for just protecting the hidden parameters.


There is a risk that we use decisions made by other projects to justify how Magnum is implemented. Heat was created 3 years ago according to https://www.openstack.org/software/project-navigator/ and Barbican only 2 years ago, thus Barbican may not have been an option (or a high risk one).

Barbican has demonstrated that the project has corporate diversity and good stability (https://www.openstack.org/software/releases/liberty/components/barbican). There are some areas that could be improved (packaging and puppet modules are often needing some more investment).

I think it is worth a go to try it out and have concrete areas to improve if there are problems.

Tim

If you don’t like code duplication between Magnum and Heat, I would suggest to move the implementation to a oslo library to make it DRY. Thoughts?

[1] https://specs.openstack.org/openstack/heat-specs/specs/juno/encrypt-hidden-parameters.html

Best regards,
Hongbin

From: David Stanek [mailto:dstanek at dstanek.com]
Sent: March-18-16 4:12 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum] High Availability


On Fri, Mar 18, 2016 at 4:03 PM Douglas Mendizábal <douglas.mendizabal at rackspace.com<mailto:douglas.mendizabal at rackspace.com>> wrote:
[snip]
>
> Regarding the Keystone solution, I'd like to hear the Keystone team's feadback on that.  It definitely sounds to me like you're trying to put a square peg in a round hole.
>

I believe that using Keystone for this is a mistake. As mentioned in the blueprint, Keystone is not encrypting the data so magnum would be on the hook to do it. So that means that if security is a requirement you'd have to duplicate more than just code. magnum would start having a larger security burden. Since we have a system designed to securely store data I think that's the best place for data that needs to be secure.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160322/29a866c7/attachment.html>