Open Stack

Hi guys,

I’ve a weird situation here.

I smoothly operate a large scale multi-region Octavia service using the
default amphora driver which imply the use of nova instances as
loadbalancers.

Everything is running really well and our customers (K8s and traditional
users) are really  happy with the solution so far.

However, yesterday one of those customers using the loadbalancer in front
of their ElasticSearch cluster poked me because this loadbalancer suddenly
passed from ONLINE/OK to ONLINE/ERROR, meaning the amphoras were no longer
available but yet the anchor/member/pool and listeners settings were still
existing.

So I investigated and found out that the loadbalancer amphoras have been
destroyed by the octavia user.

The weird part is, both the master and the backup instance have been
destroyed at the same moment by the octavia service user.

Is there specific circumstances where the octavia service could decide to
delete the instances but not the anchor/members/pool ?

It’s worrying me a bit as there is no clear way to trace why does Octavia
did take this action.

I digged within the nova and Octavia DB in order to correlate the action
but except than validating my investigation it doesn’t really help as there
are no clue of why the octavia service did trigger the deletion.

If someone have any clue or tips to give me I’ll be more than happy to
discuss this situation.

Cheers guys!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190604/885780b3/attachment.html>