Well - it seems that failure of part-013 has its root cause in failure of part-011: in part-011, KUBE_NODE_PUBLIC_IP and KUBE_NODE_IP are set. Furthermore the certificates for the access to Etcd are created; this is prerequisite for any kinda of access authorization maintained by Etcd. The ip address config items require an appropriate definition as metadata. If there is no definition of that, then internet access fails and it can also not install docker in part-013 ...
Am 01.02.2019 um 10:20 schrieb Alfredo De Luca <alfredo.deluca@gmail.com>:
thanks Feilong, clemens et all.
I going to have a look later on today and see what I can do and see.
Just a question: Does the kube master need internet access to download stuff or not?
Cheers
On Fri, Feb 1, 2019 at 3:28 AM Feilong Wang <feilong@catalyst.net.nz <mailto:feilong@catalyst.net.nz>> wrote: I'm echoing Von's comments.
From the log of cloud-init-output.log, you should be able to see below error:
Cloud-init v. 0.7.9 running 'modules:final' at Wed, 30 Jan 2019 08:33:41 +0000. Up 76.51 seconds. 2019-01-30 08:37:49,209 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-011 [1] + _prefix=docker.io/openstackmagnum/ <http://docker.io/openstackmagnum/> + atomic install --storage ostree --system --system-package no --set REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt --name heat-container-agent docker.io/openstackmagnum/heat-container-agent:queens-stable <http://docker.io/openstackmagnum/heat-container-agent:queens-stable> The docker daemon does not appear to be running. + systemctl start heat-container-agent Failed to start heat-container-agent.service: Unit heat-container-agent.service not found. 2019-01-30 08:38:10,250 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-013 [5]
Then please go to /var/lib/cloud/instances/<instance_id>/scripts to find the script 011 and 013 to run it manually to get the root cause. And welcome to pop up into #openstack-containers irc channel.
On 30/01/19 11:43 PM, Clemens Hardewig wrote:
Read the cloud-Init.log! There you can see that your /var/lib/.../part-011 part of the config script finishes with error. Check why.
Von meinem iPhone gesendet
Am 30.01.2019 um 10:11 schrieb Alfredo De Luca <alfredo.deluca@gmail.com <mailto:alfredo.deluca@gmail.com>>:
here are also the logs for the cloud init logs from the k8s master....
On Wed, Jan 30, 2019 at 9:30 AM Alfredo De Luca <alfredo.deluca@gmail.com <mailto:alfredo.deluca@gmail.com>> wrote: <image.png> In the meantime this is my cluster template
On Wed, Jan 30, 2019 at 9:17 AM Alfredo De Luca <alfredo.deluca@gmail.com <mailto:alfredo.deluca@gmail.com>> wrote: hi Clemens and Ignazio. thanks for your support. it must be network related but I don't do something special apparently to create a simple k8s cluster. I ll post later on configurations and logs as you Clemens suggested.
Cheers
On Tue, Jan 29, 2019 at 9:16 PM Clemens <clemens.hardewig@crandale.de <mailto:clemens.hardewig@crandale.de>> wrote: … an more important: check the other log cloud-init.log for error messages (not only cloud-init-output.log)
Am 29.01.2019 um 16:07 schrieb Alfredo De Luca <alfredo.deluca@gmail.com <mailto:alfredo.deluca@gmail.com>>:
Hi Ignazio and Clemens. I haven\t configure the proxy and all the logs on the kube master keep saying the following
+ '[' ok = '[-]poststarthook/bootstrap-controller failed: not finished [+]poststarthook/extensions/third-party-resources ok [-]poststarthook/rbac/bootstrap-roles failed: not finished healthz check failed' ']' + sleep 5 ++ curl --silent http://127.0.0.1:8080/healthz <http://127.0.0.1:8080/healthz> + '[' ok = '' ']' + sleep 5 ++ curl --silent http://127.0.0.1:8080/healthz <http://127.0.0.1:8080/healthz> + '[' ok = '[-]poststarthook/bootstrap-controller failed: not finished [+]poststarthook/extensions/third-party-resources ok [-]poststarthook/rbac/bootstrap-roles failed: not finished healthz check failed' ']' + sleep 5
Not sure what to do. My configuration is ... eth0 - 10.1.8.113
But the openstack configration in terms of networkin is the default from ansible-openstack which is 172.29.236.100/22 <http://172.29.236.100/22>
Maybe that's the problem?
On Tue, Jan 29, 2019 at 2:26 PM Ignazio Cassano <ignaziocassano@gmail.com <mailto:ignaziocassano@gmail.com>> wrote: Hello Alfredo, your external network is using proxy ? If you using a proxy, and yuo configured it in cluster template, you must setup no proxy for 127.0.0.1 Ignazio
Il giorno mar 29 gen 2019 alle ore 12:26 Clemens Hardewig <clemens.hardewig@crandale.de <mailto:clemens.hardewig@crandale.de>> ha scritto: At least on fedora there is a second cloud Init log as far as I remember-Look into both
Br c
Von meinem iPhone gesendet
Am 29.01.2019 um 12:08 schrieb Alfredo De Luca <alfredo.deluca@gmail.com <mailto:alfredo.deluca@gmail.com>>:
thanks Clemens. I looked at the cloud-init-output.log on the master... and at the moment is doing the following....
++ curl --silent http://127.0.0.1:8080/healthz <http://127.0.0.1:8080/healthz> + '[' ok = '' ']' + sleep 5 ++ curl --silent http://127.0.0.1:8080/healthz <http://127.0.0.1:8080/healthz> + '[' ok = '' ']' + sleep 5 ++ curl --silent http://127.0.0.1:8080/healthz <http://127.0.0.1:8080/healthz> + '[' ok = '' ']' + sleep 5
Network ....could be but not sure where to look at
On Tue, Jan 29, 2019 at 11:34 AM Clemens Hardewig <clemens.hardewig@crandale.de <mailto:clemens.hardewig@crandale.de>> wrote: Yes, you should check the cloud-init logs of your master. Without having seen them, I would guess a network issue or you have selected for your minion nodes a flavor using swap perhaps ... So, log files are the first step you could dig into... Br c Von meinem iPhone gesendet
Am 28.01.2019 um 15:34 schrieb Alfredo De Luca <alfredo.deluca@gmail.com <mailto:alfredo.deluca@gmail.com>>:
Hi all. I finally instaledl successufully openstack ansible (queens) but, after creating a cluster template I create k8s cluster, it stuck on
kube_masters <https://10.1.8.113/project/stacks/stack/6221608c-e7f1-4d76-b694-cdd7ec22c386/kube_masters/> b7204f0c-b9d8-4ef2-8f0b-afe4c077d039 <https://10.1.8.113/project/stacks/stack/b7204f0c-b9d8-4ef2-8f0b-afe4c077d039/> OS::Heat::ResourceGroup 16 minutes Create In Progress state changed create in progress....and after around an hour it says...time out. k8s master seems to be up.....at least as VM.
any idea?
Alfredo
-- Alfredo
-- Alfredo
-- Alfredo
-- Alfredo
-- Alfredo
<cloud-init.log> <cloud-init-output.log> -- Cheers & Best regards, Feilong Wang (王飞龙)
Senior Cloud Software Engineer Tel: +64-48032246 Email: flwang@catalyst.net.nz <mailto:flwang@catalyst.net.nz> Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------
-- Alfredo