[openstack-dev] [k8s][octavia][lbaas] Experiences on using the LB APIs with K8s
chris at openstack.org
Fri Mar 16 16:08:00 UTC 2018
> On Mar 16, 2018, at 7:40 AM, Simon Leinen <simon.leinen at switch.ch> wrote:
> Joe Topjian writes:
>> Terraform hat! I want to slightly nit-pick this one since the words
>> "leak" and "admin-priv" can sound scary: Terraform technically wasn't
>> doing anything wrong. The problem was that Octavia was creating
>> resources but not setting ownership to the tenant. When it came time
>> to delete the resources, Octavia was correctly refusing, though it
>> incorrectly created said resources.
> I dunno... if Octavia created those lower-layer resources on behalf of
> the user, then Octavia shouldn't refuse to remove those resources when
> the same user later asks it to - independent of what ownership Octavia
> chose to apply to those resources. (It would be different it Neutron or
> Nova were asked by the user directly to remove the resources created by
>> From reviewing the discussion, other parties were discovering this
>> issue and patching in parallel to your discovery. Both xgerman and
>> Vexxhost jumped in to confirm the behavior seen by Terraform. Vexxhost
>> quickly applied the patch. It was a really awesome collaboration
>> between yourself, dims, xgerman, and Vexxhost.
> Speaking as another operator: Does anyone seriously expect us to deploy
> a service (Octavia) in production at a stage where it exhibits this kind
> of behavior? Having to clean up leftover resources because the users who
> created them cannot remove them is not my idea of fun. (And note that
> like most operators, we're a few releases behind, so we might not even
> get access to backports IF this gets fixed.)
Simon and Joe, one thing that I was not clear on (again, goes back to the
statement that mistakes I make are my own), is that this is behavior,
admin-scoped resources being created then not released, was seen in the Neutron
LBaaSv2 service. The fix _was_ to deploy Octavia and not use the Neutron API.
As such, I'm reluctant to use Terraform (or really, any other SDK) to deploy
load balancers against the Neutron API. I don't want to be leaking a bunch of
resources I can't delete. It's not good for the apps I’m trying to run and it’s
definitely not good for the cloud provider. I have much more confidence developing
against the Octavia service.
We figured this out as a group effort between Vexxhost, Joe, and the Octavia
team, and I'm exceptionally grateful to all of them for helping me to sort
those issues out.
Now, I ultimately dropped it in my own code because I can't rely on the
existence of Octavia across all clouds. It had nothing to do with the either
the reliability of the GopherCloud/Terraform SDKs or Octavia itself.
So, to repeat, leaking admin-scoped resources is a Neutron LBaaSv2 bug,
not an Octavia bug.
> In our case we're not a compute-oriented cloud provider, and some of our
> customers would really like to have a good LBaaS as part of our IaaS
> offering. But our experience with this was so-so in the past - for
> example, we had to help customers migrate from LBaaSv1 to LBaaSv2. Our
> resources (people, tolerance to user-affecting bugs and forced upgrades
> etc.) are limited, so we've become careful.
> For users who want to use Kubernetes on our OpenStack service, we rather
> point them to Kubernetes's Ingress controller, which performs the LB
> function without requiring much from the underlying cloud. Seems like a
> fine solution.
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev