Octavia
Chris Lyons
chris at lyonsgroup.family
Mon Aug 9 01:57:49 UTC 2021
Well…it gets. A lot further…. I see this error now…. Im looking around to see if it’s a security group missing or if there is some other setting I missed. Im not seeing any scripts to prep the env…usually there is something like that if it’s a security group…anyone know?
|__Atom 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-create-amphora-indb' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': '5b4598ee-d055-40c4-997d-ea41009d819e'}, 'provides': 'b6a7018d-d436-477b-8aad-b1b53b1e6cde'}
|__Flow 'STANDALONE-octavia-create-amp-for-lb-subflow'
|__Atom 'STANDALONE-octavia-get-amphora-for-lb-subflow-octavia-mapload-balancer-to-amphora' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': '5b4598ee-d055-40c4-997d-ea41009d819e', 'flavor': {}, 'availability_zone': {}, 'server_group_id': None}, 'provides': None}
|__Flow 'STANDALONE-octavia-get-amphora-for-lb-subflow'
|__Atom 'octavia.controller.worker.v1.tasks.network_tasks.GetSubnetFromVIP' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer': <octavia.common.data_models.LoadBalancer object at 0x7f10d00d48d0>}, 'provides': <octavia.network.data_models.Subnet object at 0x7f10d0077a58>}
|__Atom 'octavia.controller.worker.v1.tasks.network_tasks.UpdateVIPSecurityGroup' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': '5b4598ee-d055-40c4-997d-ea41009d819e'}, 'provides': '3512c022-9189-4709-be49-46555d7ce8c6'}
|__Atom 'octavia.controller.worker.v1.tasks.database_tasks.UpdateVIPAfterAllocation' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': '5b4598ee-d055-40c4-997d-ea41009d819e', 'vip': <octavia.common.data_models.Vip object at 0x7f10d0118d68>}, 'provides': <octavia.common.data_models.LoadBalancer object at 0x7f10d00d48d0>}
|__Atom 'octavia.controller.worker.v1.tasks.network_tasks.AllocateVIP' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer': <octavia.common.data_models.LoadBalancer object at 0x7f10d1966b38>}, 'provides': <octavia.common.data_models.Vip object at 0x7f10d0118d68>}
|__Atom 'reload-lb-before-allocate-vip' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': '5b4598ee-d055-40c4-997d-ea41009d819e'}, 'provides': <octavia.common.data_models.LoadBalancer object at 0x7f10d1966b38>}
|__Atom 'octavia.controller.worker.v1.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'loadbalancer_id': '5b4598ee-d055-40c4-997d-ea41009d819e'}, 'provides': None}
|__Flow 'octavia-create-loadbalancer-flow': octavia.amphorae.driver_exceptions.exceptions.TimeOutException: contacting the amphora timed out
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker Traceback (most recent call last):
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker File "/usr/lib/python3.6/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker result = task.execute(**arguments)
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker File "/usr/lib/python3.6/site-packages/octavia/controller/worker/v1/tasks/amphora_driver_tasks.py", line 424, in execute
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker amp_info = self.amphora_driver.get_info(amphora)
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker File "/usr/lib/python3.6/site-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 373, in get_info
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker amphora, raise_retry_exception=raise_retry_exception)
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker File "/usr/lib/python3.6/site-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 106, in _populate_amphora_api_version
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker raise_retry_exception=raise_retry_exception)['api_version']
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker File "/usr/lib/python3.6/site-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 744, in get_api_version
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker raise_retry_exception=raise_retry_exception)
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker File "/usr/lib/python3.6/site-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 738, in request
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker raise driver_except.TimeOutException()
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker octavia.amphorae.driver_exceptions.exceptions.TimeOutException: contacting the amphora timed out
2021-08-08 21:22:35.965 27 ERROR octavia.controller.worker.v1.controller_worker
2021-08-08 21:22:35.995 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (7f8c1119-9020-4fb8-ae98-1495875a3dc2) transitioned into state 'REVERTED' from state 'REVERTING'
2021-08-08 21:22:36.003 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-info' (ca7b4681-3fe6-4834-b7ee-2fb2dbe01c3c) transitioned into state 'REVERTED' from state 'REVERTING'
2021-08-08 21:22:36.011 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-compute-wait' (2ef36bd4-15be-468f-9a8c-f6b60728da6c) transitioned into state 'REVERTED' from state 'REVERTING'
2021-08-08 21:22:36.017 27 WARNING octavia.controller.worker.v1.tasks.database_tasks [-] Reverting mark amphora booting in DB for amp id b6a7018d-d436-477b-8aad-b1b53b1e6cde and compute id c93393df-a39a-486e-812d-73652209fcff
2021-08-08 21:22:36.049 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-booting-indb' (6aac42e0-fe77-43b8-9323-e0f79304d9d6) transitioned into state 'REVERTED' from state 'REVERTING'
2021-08-08 21:22:36.056 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-computeid' (592da2b6-ab90-4292-a854-c6e2a679dabf) transitioned into state 'REVERTED' from state 'REVERTING'
2021-08-08 21:22:36.061 27 WARNING octavia.controller.worker.v1.tasks.compute_tasks [-] Reverting compute create for amphora with id b6a7018d-d436-477b-8aad-b1b53b1e6cde and compute id: c93393df-a39a-486e-812d-73652209fcff
2021-08-08 21:22:36.576 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-compute-create' (59401bf8-9191-4ac1-b4f3-b551a139df9d) transitioned into state 'REVERTED' from state 'REVERTING'
2021-08-08 21:22:36.582 27 WARNING octavia.controller.worker.v1.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' (9e731d93-8cb8-46cb-9be4-073e1579a5f8) transitioned into state 'REVERTED' from state 'REVERTING'
From: Chris Lyons <chris at lyonsgroup.family>
Date: Sunday, August 8, 2021 at 8:56 PM
To: Bernd Bausch <berndbausch at gmail.com>, openstack-discuss at lists.openstack.org <openstack-discuss at lists.openstack.org>
Subject: Re: Octavia
I think I might have solved it. It looks like during the kolla install a “Octavia-openrc.sh” script is created which sets up the env to use an account called “service”. After running that and re-uploading the amphora image I am now able to see amphora instances get created. Thank you Bernd for the suggestion that you gve! It caused me to look around in the permissions area and found the solution!
From: Bernd Bausch <berndbausch at gmail.com>
Date: Friday, August 6, 2021 at 8:21 PM
To: openstack-discuss at lists.openstack.org <openstack-discuss at lists.openstack.org>, Chris Lyons <chris at lyonsgroup.family>
Subject: Re: Octavia
I had that problem when setting up a Victoria Kolla cluster, and it turned out that the image was owned by the wrong project, as suspected by Michael. I changed the owner in Glance, and it worked. The owner must be 4269b3cd5452416e8153f5b4f5adcf0c in Chris' case (openstack image set --project 4269b3cd5452416e8153f5b4f5adcf0c amphora-image-name).
I don't think this is a Kolla problem, as Kolla leaves the responsibility of building and uploading the image to the administrator (or it did so at Victoria).
Bernd Bausch
On 2021/08/06 11:07 PM, Chris Lyons wrote:
Both of those appear ok…. I do have the amphora image set to public and I see this in the Octavia.conf file:
[controller_worker]
amp_ssh_key_name = octavia_ssh_key
amp_image_tag = amphora
amp_image_owner_id = 4269b3cd5452416e8153f5b4f5adcf0c
amp_boot_network_list = adfa01a9-55bd-42ea-b849-81060d5d7c09
amp_secgroup_list = ecf269e2-42fc-477b-9ce1-38d60c1d8d5d
amp_flavor_id = c854d1e7-885f-4f7a-88a3-79728b561830
client_ca = /etc/octavia/certs/client_ca.cert.pem
network_driver = allowed_address_pairs_driver
compute_driver = compute_nova_driver
amphora_driver = amphora_haproxy_rest_driver
amp_active_retries = 100
amp_active_wait_sec = 2
loadbalancer_topology = SINGLE
From: Michael Johnson <johnsomor at gmail.com><mailto:johnsomor at gmail.com>
Date: Friday, August 6, 2021 at 10:04 AM
To: Chris Lyons <chris at lyonsgroup.family><mailto:chris at lyonsgroup.family>
Cc: openstack-discuss at lists.openstack.org<mailto:openstack-discuss at lists.openstack.org> <openstack-discuss at lists.openstack.org><mailto:openstack-discuss at lists.openstack.org>
Subject: Re: Octavia
This is a kolla bug most likely.
This could be caused by a few possible issues in kolla:
1. The images are owned by the wrong project. Kolla may deploy
Octavia under a special service project. The image kolla uploads to
glance must be uploaded under the service project as owner. A simple
test to see if this is the issue would be to set the image as "public"
such that all users of the cloud can see it. If a load balancer can be
created after that change, the images are loaded under a project that
Octavia cannot access.
2. The other possibility is kolla configured an alternate tag name.
Check the [controller_worker] amp_image_tag setting [1] in
octavia.conf. The images must be tagged with the same name as is
configured in the octavia.conf for the controllers.
Let us know either way (and ideally open a bug for kolla) what you find out.
Michael
[1] https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_image_tag
On Fri, Aug 6, 2021 at 5:52 AM Chris Lyons <chris at lyonsgroup.family><mailto:chris at lyonsgroup.family> wrote:
>
> Octavia group,
>
>
>
> I created this story to see if I could get some assistance on an Octavia issue I am having with my infra. I did a vanilla install of Openstack Wallaby using the Kolla project with Octavia enabled and auto-configure set. Everything appeared to install fine. When I try to create a load balancer using the horizon console or the cli or through further installs such as cloudfoundry that require a loadbalancer, I get an error from glance about “No image found with tag amphora” even though the image does exist. I hope it is something simple or an oversight on my part. Could I get some ideas or places to look?
>
>
>
> Story:
>
>
>
> https://storyboard.openstack.org/#!/story/2009103
>
>
>
> Kolla install followed:
>
>
>
> https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html
>
>
>
> cli output :
>
>
>
> [root at kolla ~]# openstack image list --tag amphora
>
> +--------------------------------------+---------------------+--------+
>
> | ID | Name | Status |
>
> +--------------------------------------+---------------------+--------+
>
> | 8f7398e7-7912-43b8-8131-e90f21c91ab4 | amphora | active |
>
> | 9bf14389-8521-4fb7-a7db-925f4dea1bf3 | amphora-x64-haproxy | active |
>
> +--------------------------------------+---------------------+--------+
>
> [root at kolla ~]#
>
>
>
> My Infra scripts (relevant):
>
>
>
> https://github.com/mephmanx/openstack-scripts/blob/32363ef5753fdec381f713c747c90a5c14b3ae72/kolla.sh#L340
>
>
>
>
>
> I am admin of said infra so if anyone has time or availability (or interest) to log in and take a look I would be happy to provide creds.
>
>
>
> Thank you for any time that you could provide or offer!
This email has been scanned by Inbound Shield™.
This email has been scanned by Inbound Shield™.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210809/646fb747/attachment-0001.html>
More information about the openstack-discuss
mailing list