[nova][neutron][ptg] Summary: Leaking resources when ports are deleted out-of-band

Michael Johnson johnsomor at gmail.com
Sat May 4 16:25:08 UTC 2019


I think this will have implications for Octavia, but we can work through those.

There are cases during cleanup from an error where we delete ports
owned by "Octavia" that have not yet be attached to a nova instance.
My understanding of the above discussion is that this would not be an
issue under this change.

However....

We also, currently, manipulate the ports we have hot-plugged
(attached) to nova instances where the port "device_owner" has become
"compute:nova", mostly for failover scenarios and cases where nova
detach fails and we have to revert the action.

Now, if the "proper" new procedure is to first detach before deleting
the port, we can look at attempting that. But, in the common failure
scenarios we see nova failing to complete this, if for example the
compute host has been powered off. In this scenario we still need to
delete the neutron port for both resource cleanup and quota reasons.
This so we can create a new port and attach it to a new instance to
recover.

I think this change will impact our current port manage flows, so we
should proceed cautiously, test heavily, and potentially address some
of the nova failure scenarios at the same time.

Michael

On Fri, May 3, 2019 at 5:23 PM Akihiro Motoki <amotoki at gmail.com> wrote:
>
>
>
> On Fri, May 3, 2019 at 4:11 PM Matt Riedemann <mriedemos at gmail.com> wrote:
>>
>> On 5/3/2019 3:35 PM, Balázs Gibizer wrote:
>> > 2) Matt had a point after the session that if Neutron enforces that
>> > only unbound port can be deleted then not only Nova needs to be changed
>> > to unbound a port before delete it, but possibly other Neutron
>> > consumers (Octavia?).
>>
>> And potentially Zun, there might be others, Magnum, Heat, idk?
>>
>> Anyway, this is a thing that has been around forever which admins
>> shouldn't do, do we need to prioritize making this change in both
>> neutron and nova to make two requests to delete a bound port? Or is just
>> logging the ERROR that you've leaked allocations, tsk tsk, enough? I
>> tend to think the latter is fine until someone comes along saying this
>> is really hurting them and they have a valid use case for deleting bound
>> ports out of band from nova.
>
>
> neutron deines a special role called "advsvc"  for advanced network services [1].
> I think we can change neutron to block deletion of bound ports for regular users and
> allow users with "advsvc" role to delete bound ports.
> I haven't checked which projects currently use "advsvc".
>
> [1] https://opendev.org/openstack/neutron/src/branch/master/neutron/conf/policies/port.py#L53-L59
>
>>
>>
>> --
>>
>> Thanks,
>>
>> Matt
>>



More information about the openstack-discuss mailing list