Open Stack

Hi Pierre,

thanks for the quick reply and pointing me in the right direction. I 
used a central local docker registry in the past. Is this still the 
latest documentation:

https://docs.openstack.org/kolla-ansible/latest/user/multinode.html

I stopped using it, because I always came across problems where it was 
no longer possible to pull any container images. Just today I had this 
issue without using a local registry:

TASK [service-images-pull : common | Pull images] 
***********************************************************************************************************************************************************************************************************************************************************************************************************************************************************
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (1 retries left).
FAILED - RETRYING: common | Pull images (1 retries left).
FAILED - RETRYING: common | Pull images (1 retries left).
FAILED - RETRYING: common | Pull images (1 retries left).
FAILED - RETRYING: common | Pull images (1 retries left).
failed: [gedaopl09] (item=fluentd) => {"ansible_loop_var": "item", 
"attempts": 3, "changed": true, "item": {"key": "fluentd", "value": 
{"container_name": "fluentd", "dimensions": {}, "enabled": true, 
"environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": 
"fluentd", "image": 
"quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": 
["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", 
"/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", 
"fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most 
recent call last):\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
268, in _raise_for_status\\n response.raise_for_status()\\n  File 
\"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in 
raise_for_status\\n    raise HTTPError(http_error_msg, 
response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not 
Found for url: 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring 
handling of the above exception, another exception 
occurred:\\n\\nTraceback (most recent call last):\\n  File 
\"/tmp/ansible_kolla_docker_payload_w20pu6xy/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", 
line 381, in main\\n  File 
\"/tmp/ansible_kolla_docker_payload_w20pu6xy/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", 
line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  
File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", 
line 430, in pull\\n    self._raise_for_status(response)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
270, in _raise_for_status\\n    raise 
create_api_error_from_http_exception(e)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in 
create_api_error_from_http_exception\\n    raise cls(e, 
response=response, explanation=explanation)\\ndocker.errors.NotFound: 
404 Client Error for 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: 
Not Found (\"manifest for 
quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest 
unknown: manifest unknown\")\\n'"}
failed: [gedaopl11] (item=fluentd) => {"ansible_loop_var": "item", 
"attempts": 3, "changed": true, "item": {"key": "fluentd", "value": 
{"container_name": "fluentd", "dimensions": {}, "enabled": true, 
"environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": 
"fluentd", "image": 
"quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": 
["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", 
"/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", 
"fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most 
recent call last):\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
268, in _raise_for_status\\n response.raise_for_status()\\n  File 
\"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in 
raise_for_status\\n    raise HTTPError(http_error_msg, 
response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not 
Found for url: 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring 
handling of the above exception, another exception 
occurred:\\n\\nTraceback (most recent call last):\\n  File 
\"/tmp/ansible_kolla_docker_payload_xgz79c_7/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", 
line 381, in main\\n  File 
\"/tmp/ansible_kolla_docker_payload_xgz79c_7/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", 
line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  
File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", 
line 430, in pull\\n    self._raise_for_status(response)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
270, in _raise_for_status\\n    raise 
create_api_error_from_http_exception(e)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in 
create_api_error_from_http_exception\\n    raise cls(e, 
response=response, explanation=explanation)\\ndocker.errors.NotFound: 
404 Client Error for 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: 
Not Found (\"manifest for 
quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest 
unknown: manifest unknown\")\\n'"}
failed: [gedaopl04] (item=fluentd) => {"ansible_loop_var": "item", 
"attempts": 3, "changed": true, "item": {"key": "fluentd", "value": 
{"container_name": "fluentd", "dimensions": {}, "enabled": true, 
"environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": 
"fluentd", "image": 
"quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": 
["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", 
"/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", 
"fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most 
recent call last):\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
268, in _raise_for_status\\n response.raise_for_status()\\n  File 
\"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in 
raise_for_status\\n    raise HTTPError(http_error_msg, 
response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not 
Found for url: 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring 
handling of the above exception, another exception 
occurred:\\n\\nTraceback (most recent call last):\\n  File 
\"/tmp/ansible_kolla_docker_payload_1petvtop/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", 
line 381, in main\\n  File 
\"/tmp/ansible_kolla_docker_payload_1petvtop/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", 
line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  
File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", 
line 430, in pull\\n    self._raise_for_status(response)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
270, in _raise_for_status\\n    raise 
create_api_error_from_http_exception(e)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in 
create_api_error_from_http_exception\\n    raise cls(e, 
response=response, explanation=explanation)\\ndocker.errors.NotFound: 
404 Client Error for 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: 
Not Found (\"manifest for 
quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest 
unknown: manifest unknown\")\\n'"}
failed: [gedaopl07] (item=fluentd) => {"ansible_loop_var": "item", 
"attempts": 3, "changed": true, "item": {"key": "fluentd", "value": 
{"container_name": "fluentd", "dimensions": {}, "enabled": true, 
"environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": 
"fluentd", "image": 
"quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": 
["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", 
"/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", 
"fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most 
recent call last):\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
268, in _raise_for_status\\n response.raise_for_status()\\n  File 
\"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in 
raise_for_status\\n    raise HTTPError(http_error_msg, 
response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not 
Found for url: 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring 
handling of the above exception, another exception 
occurred:\\n\\nTraceback (most recent call last):\\n  File 
\"/tmp/ansible_kolla_docker_payload_uzj0wg5e/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", 
line 381, in main\\n  File 
\"/tmp/ansible_kolla_docker_payload_uzj0wg5e/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", 
line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  
File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", 
line 430, in pull\\n    self._raise_for_status(response)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
270, in _raise_for_status\\n    raise 
create_api_error_from_http_exception(e)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in 
create_api_error_from_http_exception\\n    raise cls(e, 
response=response, explanation=explanation)\\ndocker.errors.NotFound: 
404 Client Error for 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: 
Not Found (\"manifest for 
quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest 
unknown: manifest unknown\")\\n'"}
failed: [gedaopl10] (item=fluentd) => {"ansible_loop_var": "item", 
"attempts": 3, "changed": true, "item": {"key": "fluentd", "value": 
{"container_name": "fluentd", "dimensions": {}, "enabled": true, 
"environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": 
"fluentd", "image": 
"quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": 
["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", 
"/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", 
"fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most 
recent call last):\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
268, in _raise_for_status\\n response.raise_for_status()\\n  File 
\"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in 
raise_for_status\\n    raise HTTPError(http_error_msg, 
response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not 
Found for url: 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring 
handling of the above exception, another exception 
occurred:\\n\\nTraceback (most recent call last):\\n  File 
\"/tmp/ansible_kolla_docker_payload_a6kiyuhn/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", 
line 381, in main\\n  File 
\"/tmp/ansible_kolla_docker_payload_a6kiyuhn/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", 
line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  
File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", 
line 430, in pull\\n    self._raise_for_status(response)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 
270, in _raise_for_status\\n    raise 
create_api_error_from_http_exception(e)\\n  File 
\"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in 
create_api_error_from_http_exception\\n    raise cls(e, 
response=response, explanation=explanation)\\ndocker.errors.NotFound: 
404 Client Error for 
http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: 
Not Found (\"manifest for 
quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest 
unknown: manifest unknown\")\\n'"}
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (3 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).
FAILED - RETRYING: common | Pull images (2 retries left).

I'm on release Yoga and for some weird reason, I found the suggestion to 
change release yoga to master in /etc/kolla/globals.yaml. And this 
works. Any ideas why?

Am 02.09.2022 um 14:19 schrieb Pierre Riteau:
> Hi Oliver,
>
> The Kolla Ansible documentation describes how to configure Docker to 
> use a proxy: 
> https://docs.openstack.org/kolla-ansible/latest/reference/deployment-and-bootstrapping/bootstrap-servers.html#configuration
> This feature was introduced in Wallaby.
>
> Additionally, I suggest you configure a local Docker registry and use 
> it to fetch container images. The same documentation page describes 
> related variables.
>
> Cheers,
> Pierre Riteau (priteau)
>
> On Fri, 2 Sept 2022 at 13:33, Oliver Weinmann <oliver.weinmann at me.com> 
> wrote:
>
>     Hi,
>
>     I was wondering if there is a way to set a http proxy in
>     kolla-ansible
>     during deployment / update etc. Today I wanted to pull new containers
>     and this saturates our internet connection. What we usually do is
>     have
>     machine that massively download stuff from the internet use a squid
>     proxy that throttles their bandwidth.
>
>     I found the following feature request:
>
>     https://bugs.launchpad.net/kolla-ansible/+bug/1829586
>
>     And it seems some work had been done, but I can't seem to find any
>     info
>     if the change really made it into production:
>
>     https://review.opendev.org/c/openstack/kolla-ansible/+/692006/
>
>     Any help or hints are highly appreciated.
>
>     Cheers,
>
>     Oliver
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220902/c80bd4f1/attachment-0001.htm>