Magnum private docker registry (insecure_registry) not working?
Hi,I can't seem to get magnum (k8s) to accept my private docker registry. I wanted to have a central registry so not all hosts pull the images during deployment.For this I configured a registry:v2 docker container, pulled the images and pushed them to the local registry and added the following label to my k8s template:container_infra_prefix=172.28.7.140:4000/At first this seems to be working fine and when deploying a new k8s cluster using magnum I can see that it pulls the heat-container-agent image from my local registry:[core@k8s-admin-test-local-reg-6c4hx7gxbdhr-master-0 ~]$ sudo podman ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES2d08559b9cdc 172.28.7.140:4000/heat-container-agent:wallaby-stable-1 /usr/bin/start-he... 1 second ago Up 1 second ago heat-container-agentBut then it fails to pull the next container:tail -f /var/log/heat-config/heat-config-script/64d35aad-5453-4da4-97c7-45abb640fc90-k8s-admin-test-local-reg-6c4hx7gxbdhr-kube_masters-h3wbcqgm6qv4-0-sfagopiu52se-master_config-2f5lhvr32z7j.logWARNING Attempt 8: Trying to install kubectl. Sleeping 5s+ ssh -F /srv/magnum/.ssh/config root@localhost '/usr/bin/podman run --entrypoint /bin/bash --name install-kubectl --net host --privileged --rm --user root --volume /srv/magnum/bin:/host/srv/magnum/bin 172.28.7.140:4000/hyperkube:v1.23.3-rancher1 -c '\''cp /usr/local/bin/kubectl /host/srv/magnum/bin/kubectl'\'''Trying to pull 172.28.7.140:4000/hyperkube:v1.23.3-rancher1...Error: initializing source docker://172.28.7.140:4000/hyperkube:v1.23.3-rancher1: pinging container registry 172.28.7.140:4000: Get "https://172.28.7.140:4000/v2/": http: server gave HTTP response to HTTPS clientI don't know why but there is no /etc/docker/daemon.json and the /etc/sysconfig/docker also doesn'T contain the line for my insecure registry:root@k8s-admin-test-local-reg-6c4hx7gxbdhr-master-0 ~]# cat /etc/sysconfig/docker# /etc/sysconfig/docker# Modify these options if you want to change the way the docker daemon runsOPTIONS="--selinux-enabled \ --log-driver=journald \ --live-restore \ --default-ulimit nofile=1024:1024 \ --init-path /usr/libexec/docker/docker-init \ --userland-proxy-path /usr/libexec/docker/docker-proxy \"As soon as I manually add my insecure registry here it works just fine. I looked at the magnum code and there is indeed some lines that should actually handle this, but it doesn't seem to be working. What is also weird is that while there is the Option in the Horizon WebUI to set an insecure registry, the openstack coe command doesn't offer this.Best Regards,Oliver
Hi all,Problem solved. I was not using the latest fedora Core is 35 image. It is kind of hard to find it since the last version on the page is 36 and there is no download archive. I was able to find a Reddit post (https://www.reddit.com/r/Fedora/comments/mmtv5c/is_there_an_archive_for_prev...) on how to download older versions. Using the latest fedora core os 35 version, it works just fine. Still I have not found a way to set the insecure-registry via cmdline. I saw the option when using terraform.Cheers,OliverVon meinem iPhone gesendetAm 16.12.2022 um 13:08 schrieb Oliver Weinmann <oliver.weinmann@me.com>:Hi,I can't seem to get magnum (k8s) to accept my private docker registry. I wanted to have a central registry so not all hosts pull the images during deployment.For this I configured a registry:v2 docker container, pulled the images and pushed them to the local registry and added the following label to my k8s template:container_infra_prefix=172.28.7.140:4000/At first this seems to be working fine and when deploying a new k8s cluster using magnum I can see that it pulls the heat-container-agent image from my local registry:[core@k8s-admin-test-local-reg-6c4hx7gxbdhr-master-0 ~]$ sudo podman ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES2d08559b9cdc 172.28.7.140:4000/heat-container-agent:wallaby-stable-1 /usr/bin/start-he... 1 second ago Up 1 second ago heat-container-agentBut then it fails to pull the next container:tail -f /var/log/heat-config/heat-config-script/64d35aad-5453-4da4-97c7-45abb640fc90-k8s-admin-test-local-reg-6c4hx7gxbdhr-kube_masters-h3wbcqgm6qv4-0-sfagopiu52se-master_config-2f5lhvr32z7j.logWARNING Attempt 8: Trying to install kubectl. Sleeping 5s+ ssh -F /srv/magnum/.ssh/config root@localhost '/usr/bin/podman run --entrypoint /bin/bash --name install-kubectl --net host --privileged --rm --user root --volume /srv/magnum/bin:/host/srv/magnum/bin 172.28.7.140:4000/hyperkube:v1.23.3-rancher1 -c '\''cp /usr/local/bin/kubectl /host/srv/magnum/bin/kubectl'\'''Trying to pull 172.28.7.140:4000/hyperkube:v1.23.3-rancher1...Error: initializing source docker://172.28.7.140:4000/hyperkube:v1.23.3-rancher1: pinging container registry 172.28.7.140:4000: Get "https://172.28.7.140:4000/v2/": http: server gave HTTP response to HTTPS clientI don't know why but there is no /etc/docker/daemon.json and the /etc/sysconfig/docker also doesn'T contain the line for my insecure registry:root@k8s-admin-test-local-reg-6c4hx7gxbdhr-master-0 ~]# cat /etc/sysconfig/docker# /etc/sysconfig/docker# Modify these options if you want to change the way the docker daemon runsOPTIONS="--selinux-enabled \ --log-driver=journald \ --live-restore \ --default-ulimit nofile=1024:1024 \ --init-path /usr/libexec/docker/docker-init \ --userland-proxy-path /usr/libexec/docker/docker-proxy \"As soon as I manually add my insecure registry here it works just fine. I looked at the magnum code and there is indeed some lines that should actually handle this, but it doesn't seem to be working. What is also weird is that while there is the Option in the Horizon WebUI to set an insecure registry, the openstack coe command doesn't offer this.Best Regards,Oliver
participants (1)
-
Oliver Weinmann