[Octavia][Victoria] No service listening on port 9443 in the amphora instance

Michael Johnson johnsomor at gmail.com
Wed May 5 18:25:32 UTC 2021

Hi Luke.

Yes, the amphora-agent will listen on 9443 in the amphorae instances.
It uses TLS mutual authentication, so you can get a TLS response, but
it will not let you into the API without a valid certificate. A simple
"openssl s_client" is usually enough to prove that it is listening and
requesting the client certificate.

I can't talk to the "openstack-octavia-diskimage-create" package you
found in centos, but I can discuss how to build an amphora image using
the OpenStack tools.

If you get Octavia from git or via a release tarball, we provide a
script to build the amphora image. This is how we build our images for
the testing gates, etc. and is the recommended way (at least from the
OpenStack Octavia community) to create amphora images.


For CentOS 8, the command would be:

diskimage-create.sh -g stable/victoria -i centos-minimal -d 8 -s 3 (3
is the minimum disk size for centos images, you may want more if you
are not offloading logs)

I just did a run on a fresh centos 8 instance:
git clone https://opendev.org/openstack/octavia
python3 -m venv dib
source dib/bin/activate
pip3 install diskimage-builder PyYAML six
sudo dnf install yum-utils
./diskimage-create.sh -g stable/victoria -i centos-minimal -d 8 -s 3

This built an image.

Off and on we have had issues building CentOS images due to issues in
the tools we rely on. If you run into issues with this image, drop us
a note back.


On Wed, May 5, 2021 at 9:37 AM Luke Camilleri
<luke.camilleri at zylacomputing.com> wrote:
> Hi there, i am trying to get Octavia running on a Victoria deployment on
> CentOS 8. It was a bit rough getting to the point to launch an instance
> mainly due to the load-balancer management network and the lack of
> documentation
> (https://docs.openstack.org/octavia/victoria/install/install.html) to
> deploy this oN CentOS. I will try to fix this once I have my deployment
> up and running to help others on the way installing and configuring this :-)
> At this point a LB can be launched by the tenant and the instance is
> spawned in the Octavia project and I can ping and SSH into the amphora
> instance from the Octavia node where the octavia-health-manager service
> is running using the IP within the same subnet of the amphoras
> (
> Unfortunately I keep on getting these errors in the log file of the
> worker log (/var/log/octavia/worker.log):
> 2021-05-05 01:54:49.368 14521 WARNING
> octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect
> to instance. Retrying.: requests.exceptions.ConnectionError:
> HTTPSConnectionPool(host='', p
> ort=9443): Max retries exceeded with url: // (Caused by
> NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object
> at 0x7f83e0181550>: Failed to establish a new connection: [Errno 111]
> Connection ref
> used',))
> 2021-05-05 01:54:54.374 14521 ERROR
> octavia.amphorae.drivers.haproxy.rest_api_driver [-] Connection retries
> (currently set to 120) exhausted.  The amphora is unavailable. Reason:
> HTTPSConnectionPool(host='172.16
> .4.46', port=9443): Max retries exceeded with url: // (Caused by
> NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object
> at 0x7f83e0181550>: Failed to establish a new connection: [Errno 111] Conne
> ction refused',))
> 2021-05-05 01:54:54.374 14521 ERROR
> octavia.controller.worker.v1.tasks.amphora_driver_tasks [-] Amphora
> compute instance failed to become reachable. This either means the
> compute driver failed to fully boot the
> instance inside the timeout interval or the instance is not reachable
> via the lb-mgmt-net.:
> octavia.amphorae.driver_exceptions.exceptions.TimeOutException:
> contacting the amphora timed out
> obviously the instance is deleted then and the task fails from the
> tenant's perspective.
> The main issue here is that there is no service running on port 9443 on
> the amphora instance. I am assuming that this is in fact the
> amphora-agent service that is running on the instance which should be
> listening on this port 9443 but the service does not seem to be up or
> not installed at all.
> To create the image I have installed the CentOS package
> "openstack-octavia-diskimage-create" which provides the utility
> disk-image-create but from what I can conclude the amphora-agent is not
> being installed (thought this was done automatically by default :-( )
> Can anyone let me know if the amphora-agent is what gets queried on port
> 9443 ?
> If the agent is not installed/injected by default when building the
> amphora image?
> The command to inject the amphora-agent into the amphora image when
> using the disk-image-create command?
> Thanks in advance for any assistance

More information about the openstack-discuss mailing list