Open Stack

Hi Pierre,

Right I messed it up just today. Copy pasted the command from a kolla page about updating and this refers to master. 

One more question about the variables for http proxy. They have to be in /etc/kolla/globals.yml? In case I want to go with the central local docker repo. I have run the steps to install a local docker repo on my seed node. The node from where I run kolla-ansible to deploy Openstack. Do I need to run a bootstrap for all-in-one so that the proxy setting is configured?

Von meinem iPhone gesendet

> Am 02.09.2022 um 18:52 schrieb Pierre Riteau <pierre at stackhpc.com>:
> 
> 
> Hi Oliver,
> 
> I think you have a mismatch of your kolla-ansible version here. If you look at the fluent image that it is trying to pull, you will see that it has only master-* tags: https://quay.io/repository/openstack.kolla/fluentd?tab=tags&tag=latest
> 
> This is because in the development version of kolla-ansible (currently the master branch, which will be released as zed later this year) a new image naming scheme is used.
> 
> You need to switch your kolla-ansible checkout to the stable/yoga branch and reinstall it to fetch the right image, which will be something like centos-source-fluentd:yoga as seen here: https://quay.io/repository/openstack.kolla/centos-source-fluentd?tab=tags&tag=latest
> 
> Cheers,
> Pierre Riteau (priteau)
> 
>> On Fri, 2 Sept 2022 at 17:49, Oliver Weinmann <oliver.weinmann at me.com> wrote:
>> Hi Pierre,
>> 
>> thanks for the quick reply and pointing me in the right direction. I used a central local docker registry in the past. Is this still the latest documentation:
>> 
>> https://docs.openstack.org/kolla-ansible/latest/user/multinode.html
>> 
>> I stopped using it, because I always came across problems where it was no longer possible to pull any container images. Just today I had this issue without using a local registry: 
>> 
>> TASK [service-images-pull : common | Pull images] ***********************************************************************************************************************************************************************************************************************************************************************************************************************************************************
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (1 retries left).
>> FAILED - RETRYING: common | Pull images (1 retries left).
>> FAILED - RETRYING: common | Pull images (1 retries left).
>> FAILED - RETRYING: common | Pull images (1 retries left).
>> FAILED - RETRYING: common | Pull images (1 retries left).
>> failed: [gedaopl09] (item=fluentd) => {"ansible_loop_var": "item", "attempts": 3, "changed": true, "item": {"key": "fluentd", "value": {"container_name": "fluentd", "dimensions": {}, "enabled": true, "environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": "fluentd", "image": "quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": ["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", "fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most recent call last):\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 268, in _raise_for_status\\n    response.raise_for_status()\\n  File \"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/tmp/ansible_kolla_docker_payload_w20pu6xy/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n  File \"/tmp/ansible_kolla_docker_payload_w20pu6xy/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", line 430, in pull\\n    self._raise_for_status(response)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 270, in _raise_for_status\\n    raise create_api_error_from_http_exception(e)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest unknown: manifest unknown\")\\n'"}
>> failed: [gedaopl11] (item=fluentd) => {"ansible_loop_var": "item", "attempts": 3, "changed": true, "item": {"key": "fluentd", "value": {"container_name": "fluentd", "dimensions": {}, "enabled": true, "environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": "fluentd", "image": "quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": ["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", "fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most recent call last):\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 268, in _raise_for_status\\n    response.raise_for_status()\\n  File \"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/tmp/ansible_kolla_docker_payload_xgz79c_7/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n  File \"/tmp/ansible_kolla_docker_payload_xgz79c_7/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", line 430, in pull\\n    self._raise_for_status(response)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 270, in _raise_for_status\\n    raise create_api_error_from_http_exception(e)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest unknown: manifest unknown\")\\n'"}
>> failed: [gedaopl04] (item=fluentd) => {"ansible_loop_var": "item", "attempts": 3, "changed": true, "item": {"key": "fluentd", "value": {"container_name": "fluentd", "dimensions": {}, "enabled": true, "environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": "fluentd", "image": "quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": ["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", "fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most recent call last):\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 268, in _raise_for_status\\n    response.raise_for_status()\\n  File \"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/tmp/ansible_kolla_docker_payload_1petvtop/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n  File \"/tmp/ansible_kolla_docker_payload_1petvtop/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", line 430, in pull\\n    self._raise_for_status(response)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 270, in _raise_for_status\\n    raise create_api_error_from_http_exception(e)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest unknown: manifest unknown\")\\n'"}
>> failed: [gedaopl07] (item=fluentd) => {"ansible_loop_var": "item", "attempts": 3, "changed": true, "item": {"key": "fluentd", "value": {"container_name": "fluentd", "dimensions": {}, "enabled": true, "environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": "fluentd", "image": "quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": ["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", "fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most recent call last):\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 268, in _raise_for_status\\n    response.raise_for_status()\\n  File \"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/tmp/ansible_kolla_docker_payload_uzj0wg5e/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n  File \"/tmp/ansible_kolla_docker_payload_uzj0wg5e/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", line 430, in pull\\n    self._raise_for_status(response)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 270, in _raise_for_status\\n    raise create_api_error_from_http_exception(e)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest unknown: manifest unknown\")\\n'"}
>> failed: [gedaopl10] (item=fluentd) => {"ansible_loop_var": "item", "attempts": 3, "changed": true, "item": {"key": "fluentd", "value": {"container_name": "fluentd", "dimensions": {}, "enabled": true, "environment": {"KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS"}, "group": "fluentd", "image": "quay.io/openstack.kolla/fluentd:yoga-centos-stream8", "volumes": ["/etc/kolla/fluentd/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "", "kolla_logs:/var/log/kolla/", "fluentd_data:/var/lib/fluentd/data/"]}}, "msg": "'Traceback (most recent call last):\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 268, in _raise_for_status\\n    response.raise_for_status()\\n  File \"/usr/lib/python3.6/site-packages/requests/models.py\", line 940, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/tmp/ansible_kolla_docker_payload_a6kiyuhn/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", line 381, in main\\n  File \"/tmp/ansible_kolla_docker_payload_a6kiyuhn/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 451, in pull_image\\n    repository=image, tag=tag, stream=True\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/image.py\", line 430, in pull\\n    self._raise_for_status(response)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/api/client.py\", line 270, in _raise_for_status\\n    raise create_api_error_from_http_exception(e)\\n  File \"/usr/local/lib/python3.6/site-packages/docker/errors.py\", line 31, in create_api_error_from_http_exception\\n    raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=yoga-centos-stream8&fromImage=quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:yoga-centos-stream8 not found: manifest unknown: manifest unknown\")\\n'"}
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (3 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> FAILED - RETRYING: common | Pull images (2 retries left).
>> 
>> I'm on release Yoga and for some weird reason, I found the suggestion to change release yoga to master in /etc/kolla/globals.yaml. And this works. Any ideas why?
>> 
>> Am 02.09.2022 um 14:19 schrieb Pierre Riteau:
>>> Hi Oliver,
>>> 
>>> The Kolla Ansible documentation describes how to configure Docker to use a proxy: https://docs.openstack.org/kolla-ansible/latest/reference/deployment-and-bootstrapping/bootstrap-servers.html#configuration
>>> This feature was introduced in Wallaby.
>>> 
>>> Additionally, I suggest you configure a local Docker registry and use it to fetch container images. The same documentation page describes related variables.
>>> 
>>> Cheers,
>>> Pierre Riteau (priteau)
>>> 
>>> On Fri, 2 Sept 2022 at 13:33, Oliver Weinmann <oliver.weinmann at me.com> wrote:
>>>> Hi,
>>>> 
>>>> I was wondering if there is a way to set a http proxy in kolla-ansible 
>>>> during deployment / update etc. Today I wanted to pull new containers 
>>>> and this saturates our internet connection. What we usually do is have 
>>>> machine that massively download stuff from the internet use a squid 
>>>> proxy that throttles their bandwidth.
>>>> 
>>>> I found the following feature request:
>>>> 
>>>> https://bugs.launchpad.net/kolla-ansible/+bug/1829586
>>>> 
>>>> And it seems some work had been done, but I can't seem to find any info 
>>>> if the change really made it into production:
>>>> 
>>>> https://review.opendev.org/c/openstack/kolla-ansible/+/692006/
>>>> 
>>>> Any help or hints are highly appreciated.
>>>> 
>>>> Cheers,
>>>> 
>>>> Oliver
>>>> 
>>>> 
>>>> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220902/0e31f154/attachment-0001.htm>