[openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS Integration Ideas

Douglas Mendizabal douglas.mendizabal at RACKSPACE.COM
Tue Jun 10 21:26:33 UTC 2014


I think that having Barbican decide whether the user is or isn’t allowed to
delete a secret that they own based on a reference count that is not
directly controlled by them is unacceptable.   This is indeed policy
enforcement, and we’d rather not go down that path.

I’m opposed to the idea of reference counting altogether, but a couple of
other Barbican-core members are open to it, as long as it does not affect
the delete behaviors.

-Doug M.

From:  Adam Harwell <adam.harwell at RACKSPACE.COM>
Reply-To:  "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org>
Date:  Tuesday, June 10, 2014 at 4:17 PM
To:  "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org>
Subject:  Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
Integration Ideas

Doug: Right, we actually have a blueprint draft for EXACTLY this, but the
Barbican team gave us a flat "not happening, we reject this change" on
causing a delete to fail. The shadow-copy solution I proposed only came
about because the option you are proposing is not possible. :(

I also realized that really, this whole thing is an issue for the backend,
not really for the API itself — the LBaaS API will be retrieving the
key/cert from Barbican and passing it to the backend, and the backend it
what's responsible for handling it from that point (F5, Stingray etc would
never actually call back to Barbican). So, really, the Service-VM solution
we're architecting is where the shadow-copy solution needs to live, at which
point it no longer is really an issue we'd need to discuss on this mailing
list, I think. Stephen, does that make sense to you?
--Adam

https://keybase.io/rm_you


From: Doug Wiegley <dougw at a10networks.com>
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org>
Date: Tuesday, June 10, 2014 4:10 PM
To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
Integration Ideas

> A third option, that is neither shadow copying nor policy enforcement:
> 
> Ask the Barbican team to put in a small api that is effectively, “hey, I’m
> using this container” and “hey, I’m done with this container”, and the have
> their delete fail if someone is still using it.  This isn’t calling into other
> services, it’s simply getting informed of who’s using what, and not stomping
> it.  That seems pretty core to me, and the workarounds if we can’t trust the
> store to store things are pretty hacky.
> 
> Doug
> 
> 
> From: Adam Harwell <adam.harwell at RACKSPACE.COM>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Date: Tuesday, June 10, 2014 at 3:04 PM
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
> Integration Ideas
> 
> Right, service VMs are the biggest case for this, because then we WILL need to
> be tracking the barbicanID even in the backend. I also agree that it would be
> more useful for OpenStack as a whole if it were managed by a central service
> (i.e., Barbican handles this issue) rather than having to duplicate all of
> this logic in every service that utilizes the containers (VPN/FW would have to
> use essentially the same strategy, or else fragment and do something entirely
> different — the first of which is a lot of duplicated effort, and the second
> is just generally bad, already way too much fragmentation going on). On the
> other hand, the Barbican team is very opposed to doing policy enforcement
> within Barbican, and I can't say I fault them for that opinion (Barbican was
> never designed to include a policy enforcement engine). The shadow-copy
> strategy is the best alternative I can think of given the current
> project/political landscape. :(
> 
> --Adam
> 
> https://keybase.io/rm_you
> 
> 
> From: Doug Wiegley <dougw at a10networks.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Date: Tuesday, June 10, 2014 3:42 PM
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
> Integration Ideas
> 
>>> > Doug: The reasons a LB might be reprovisioned are fairly important —
>>> mostly around HA, for fail overs or capacity — exactly the times we're
>>> trying avoid a failure.
>> 
>> Certainly the ticking time bomb is a bad idea, but HA seems cleaner to do in
>> the backend, rather than at the openstack API level (the dangling reference
>> doesn’t kick in until the lbaas api is used to accomplish that failover.)
>> And the lbaas api also doesn’t have any provisions for helping to shuffle for
>> capacity, so that also becomes a backend issue.  And the backend won’t be
>> natively dealing with a barbican reference.
>> 
>> However, couple this with service VM’s, and I guess the issue pops back into
>> the forefront.  This is going to be an issue that everyone using ssl certs in
>> barbican is going to have, so it seems we’re applying a band-aid in the wrong
>> place.
>> 
>> Doug
>> 
>> 
>> From: Adam Harwell <adam.harwell at RACKSPACE.COM>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Date: Tuesday, June 10, 2014 at 2:19 PM
>> To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
>> Integration Ideas
>> 
>> Doug: The reasons a LB might be reprovisioned are fairly important — mostly
>> around HA, for fail overs or capacity — exactly the times we're trying avoid
>> a failure.
>> 
>> Stephen: yes, I am talking about storing the copy in a non-tenant way (on the
>> tenant-id for the LBaaS Service Account, not visible to the user). We would
>> have to delete our shadow-copy when the loadbalancer was updated with a new
>> barbicanID by the user, and make a copy of the new container to take its
>> place.
>> Also, yes, I think it would be quite surprising (and far from ideal) when the
>> LB you set up breaks weeks or months later when an HA event occurs and you
>> haven't actually made any "changes" in quite a long time. Unfortunately,
>> "making the key unusable in all other contexts" on a Barbican delete isn't
>> really an option, so this is the best fallback I can think of.
>> 
>> --Adam
>> 
>> https://keybase.io/rm_you
>> 
>> 
>> From: Doug Wiegley <dougw at a10networks.com>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Date: Tuesday, June 10, 2014 2:53 PM
>> To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
>> Integration Ideas
>> 
>>>> > In any case, it strikes me as misleading to have an explicit delete
>>>> command sent to Barbican not have the effect of making the key unusable in
>>>> all other contexts. It would be less surprising behavior, IMO, to have a
>>>> deleted barbican container result in connected load balancing services
>>>> breaking. (Though without notification to LBaaS, the connected service
>>>> might break weeks or months after the key disappeared from barbican, which
>>>> would be more surprising behavior.)
>>> 
>>> Since a delete in barbican will not affect neutron/lbaas, and since most
>>> backends will have had to make their own copy of the key at lb provision
>>> time, the barbican delete will not result in lbaas breaking, I think.  The
>>> shadow copy would only get used if the lb had to be re-provisioned for some
>>> reason before it was given a new key id, which seems a fair bit of
>>> complexity for what is gained.
>>> 
>>> doug
>>> 
>>> 
>>> From: Stephen Balukoff <sbalukoff at bluebox.net>
>>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>>> <openstack-dev at lists.openstack.org>
>>> Date: Tuesday, June 10, 2014 at 1:47 PM
>>> To: "OpenStack Development Mailing List (not for usage questions)"
>>> <openstack-dev at lists.openstack.org>
>>> Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
>>> Integration Ideas
>>> 
>>> Adam-- 
>>> 
>>> Wouldn't the user see the duplicate key/cert copy in their barbican
>>> interface, or are you proposing storing these secrets in a
>>> not-assigned-to-the-tenant kind of way?
>>> 
>>> In any case, it strikes me as misleading to have an explicit delete command
>>> sent to Barbican not have the effect of making the key unusable in all other
>>> contexts. It would be less surprising behavior, IMO, to have a deleted
>>> barbican container result in connected load balancing services breaking.
>>> (Though without notification to LBaaS, the connected service might break
>>> weeks or months after the key disappeared from barbican, which would be more
>>> surprising behavior.)
>>> 
>>> Personally, I like your idea, as I think most of our users would rather have
>>> LBaaS issue warnings when the user has done something stupid like this
>>> rather than break entirely. I know our support staff would rather it behaved
>>> this way.
>>> 
>>> What's your proposal for cleaning up copied secrets when they're actually no
>>> longer in use by any LB?
>>> 
>>> Stephen
>>> 
>>> 
>>> On Tue, Jun 10, 2014 at 12:04 PM, Adam Harwell <adam.harwell at rackspace.com>
>>> wrote:
>>>> So, it looks like any sort of validation on Deletes in Barbican is going
>>>> to be a no-go. I'd like to propose a third option, which might be the
>>>> safest route to take for LBaaS while still providing some of the
>>>> convenience of using Barbican as a central certificate store. Here is a
>>>> diagram of the interaction sequence to create a loadbalancer:
>>>> http://bit.ly/1pgAC7G
>>>> 
>>>> Summary: Pass the Barbican "TLS Container ID" to the LBaaS create call,
>>>> get the container from Barbican, and store a "shadow-copy" of the
>>>> container again in Barbican, this time on the LBaaS service account.
>>>> The secret will now be duplicated (it still exists on the original tenant,
>>>> but also exists on the LBaaS tenant), but we're not talking about a huge
>>>> amount of data here -- just a few kilobytes. With this approach, we retain
>>>> most of the advantages we wanted to get from using Barbican -- we don't
>>>> need to worry about taking secret data through the LBaaS API (we still
>>>> just take a barbicanID from the user), and the user can still use a single
>>>> barbicanID (the original one they created -- the copies are invisible to
>>>> them) when passing their TLS info to other services. We gain the
>>>> additional advantage that it no longer matters what happens to the
>>>> original TLS container -- it could be deleted and it would not impact our
>>>> service.
>>>> 
>>>> What do you guys think of that option?
>>>> 
>>>> 
>>>> 
>>>> As an addendum (not saying we necessarily want to do this, but it's an
>>>> option), we COULD retain both the original and the copied barbicanID in
>>>> our database and attempt to fetch them in that order when we need the TLS
>>>> info again, which would allow us to do some alerting if the user does
>>>> delete their key. For example: the user has deleted their key because it
>>>> was compromised, but they forgot they used it on their loadbalancer -> a
>>>> failover event occurs and we attempt to fetch the info from Barbican ->
>>>> the first fetch fails, but the second fetch to our copy succeeds -> the
>>>> failover of the LB is successful, and we send an alert to notify the user
>>>> that their LB is using TLS info that has been deleted from Barbican.
>>>> 
>>>> 
>>>>         --Adam
>>>> 
>>>> 
>>>> https://keybase.io/rm_you
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 6/10/14 6:21 AM, "Clark, Robert Graham" <robert.clark at hp.com> wrote:
>>>> 
>>>>> >It looks like this has come full circle and we are back at the simplest
>>>>> >case.
>>>>> >
>>>>> ># Containers are immutable
>>>>> ># Changing a cert means creating a new container and, when ready,
>>>>> >pointing LBaaS at the new container
>>>>> >
>>>>> >This makes a lot of sense to me, it removes a lot of handholding and
>>>>> >keeps Barbican and LBaaS nicely decoupled. It also keeps certificate
>>>>> >lifecycle management firmly in the hands of the user, which imho is a
>>>>> >good thing. With this model it¹s fairly trivial to provide guidance /
>>>>> >additional tooling for lifecycle management if required but at the same
>>>>> >time the simplest case (I want a cert and I want LBaaS) is met without
>>>>> >massive code overhead for edge-cases.
>>>>> >
>>>>> >
>>>>> >From: Vijay Venkatachalam
>>>>> ><Vijay.Venkatachalam at citrix.com<mailto:Vijay.Venkatachalam at citrix.com>>
>>>>> >Reply-To: OpenStack List
>>>>> 
><openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.o>>>>>
r
>>>>> >g>>
>>>>> >Date: Tuesday, 10 June 2014 05:48
>>>>> >To: OpenStack List
>>>>> 
><openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.o>>>>>
r
>>>>> >g>>
>>>>> >Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
>>>>> >Integration Ideas
>>>>> >
>>>>> >
>>>>> >My vote is for option #2 (without the registration). It is simpler to
>>>>> >start with this approach. How is delete handled though?
>>>>> >
>>>>> >Ex. What is the expectation when user attempts to delete a
>>>>> >certificate/container which is referred by an entity like LBaaS listener?
>>>>> >
>>>>> >
>>>>> >1.       Will there be validation in Barbican to prevent this? *OR*
>>>>> >
>>>>> >2.       LBaaS listener will have a dangling reference/pointer to
>>>>> >certificate?
>>>>> >
>>>>> >Thanks,
>>>>> >Vijay V.
>>>>> >
>>>>> >From: Stephen Balukoff [mailto:sbalukoff at bluebox.net]
>>>>> >Sent: Tuesday, June 10, 2014 7:43 AM
>>>>> >To: OpenStack Development Mailing List (not for usage questions)
>>>>> >Subject: Re: [openstack-dev] [Neutron][LBaaS] Barbican Neutron LBaaS
>>>>> >Integration Ideas
>>>>> >
>>>>> >Weighing in here:
>>>>> >
>>>>> >I'm all for option #2 as well.
>>>>> >
>>>>> >Stephen
>>>>> >
>>>>> >On Mon, Jun 9, 2014 at 4:42 PM, Clint Byrum
>>>>> ><clint at fewbar.com<mailto:clint at fewbar.com>> wrote:
>>>>> >Excerpts from Douglas Mendizabal's message of 2014-06-09 16:08:02 -0700:
>>>>>> >> Hi all,
>>>>>> >>
>>>>>> >> I¹m strongly in favor of having immutable TLS-typed containers, and
>>>>>> very
>>>>>> >> much opposed to storing every revision of changes done to a container.
>>>>>> >>I
>>>>>> >> think that storing versioned containers would add too much complexity
to
>>>>>> >> Barbican, where immutable containers would work well.
>>>>>> >>
>>>>> >Agree completely. Create a new one for new values. Keep the old ones
>>>>> >while they're still active.
>>>>> >
>>>>>> >>
>>>>>> >> I¹m still not sold on the idea of registering services with Barbican,
>>>>>> >>even
>>>>>> >> though (or maybe especially because) Barbican would not be using this
>>>>>> >>data
>>>>>> >> for anything.  I understand the problem that we¹re trying to solve by
>>>>>> >> associating different resources across projects, but I don¹t feel like
>>>>>> >> Barbican is the right place to do this.
>>>>>> >>
>>>>> >Agreed also, this is simply not Barbican or Neutron's role. Be a REST
>>>>> >API for secrets and networking, not all dancing all singing nannies that
>>>>> >prevent any possibly dangerous behavior with said API's.
>>>>> >
>>>>>> >> It seems we¹re leaning towards option #2, but I would argue that
>>>>>> >> orchestration of services is outside the scope of Barbican¹s role as a
>>>>>> >> secret-store.  I think this is a problem that may need to be solved at
a
>>>>>> >> higher level.  Maybe an openstack-wide registry of dependend entities
>>>>>> >> across services?
>>>>> >An optional openstack-wide registry of depended entities is called
>>>>> >"Heat".
>>>>> >
>>>>> >_______________________________________________
>>>>> >OpenStack-dev mailing list
>>>>> 
>OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.or>>>>>
g
>>>>>> >>
>>>>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >
>>>>> >
>>>>> >
>>>>> >--
>>>>> >Stephen Balukoff
>>>>> >Blue Box Group, LLC
>>>>> >(800)613-4305 x807 <tel:%28800%29613-4305%20x807>
>>>>> >
>>>>> >_______________________________________________
>>>>> >OpenStack-dev mailing list
>>>>> >OpenStack-dev at lists.openstack.org
>>>>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> 
>>> 
>>> -- 
>>> Stephen Balukoff
>>> Blue Box Group, LLC
>>> (800)613-4305 x807


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140610/f397b1ca/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5660 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140610/f397b1ca/attachment.bin>


More information about the OpenStack-dev mailing list