Open Stack

Hi Bharat / Feilong,

The problem look like in ubuntu wallaby repo's magnum DEB package. The
package does not have all the changes that a wallaby release should
contain. Like heat_container_agent_tag that should be wallaby-stable-1. In
that package it was still pointing to victoria-dev.

I have cloned stable/wallaby from https://opendev.org/openstack/magnum.git
and replaced magnum directory in /lib/python3/dist-packages with downloaded
files in stable/magnum branch. Now everything is working a expected.

[root at k8s-cluster-2zcsd5n6qnre-master-0 ~]# kubectl get nodes
NAME                                STATUS   ROLES    AGE     VERSION
k8s-cluster-2zcsd5n6qnre-master-0   Ready    master   5m43s   v1.19.10
k8s-cluster-2zcsd5n6qnre-node-0     Ready    <none>   39s     v1.19.10
[root at k8s-cluster-2zcsd5n6qnre-master-0 ~]# kubectl get pods -n kube-system
NAME                                         READY   STATUS    RESTARTS
AGE
coredns-57999d5467-2l46z                     1/1     Running   0
 5m43s
coredns-57999d5467-rn8d9                     1/1     Running   0
 5m43s
csi-cinder-controllerplugin-0                5/5     Running   0
 5m36s
csi-cinder-nodeplugin-nmjnb                  2/2     Running   0
 30s
dashboard-metrics-scraper-7b59f7d4df-mzmb6   1/1     Running   0
 5m41s
k8s-keystone-auth-knhk7                      1/1     Running   0
 5m39s
kube-dns-autoscaler-f57cd985f-7dqj5          1/1     Running   0
 5m43s
kube-flannel-ds-8d7pc                        1/1     Running   0
 5m42s
kube-flannel-ds-pcncq                        1/1     Running   0
 60s
kubernetes-dashboard-7fb447bf79-x6kvj        1/1     Running   0
 5m41s
npd-hcw25                                    1/1     Running   0
 30s
openstack-cloud-controller-manager-x759f     1/1     Running   0
 5m45s

- Ammad

On Mon, May 3, 2021 at 11:13 PM Bharat Kunwar <bharat at stackhpc.com> wrote:

> Can you try the calico plugin? The flannel plug-in has been unmaintained
> for a while.
>
> Sent from my iPhone
>
> On 3 May 2021, at 18:25, feilong <feilong at catalyst.net.nz> wrote:
>
> 
>
> Hi Ammad,
>
> What's the error of your kubelet? If the node is in not ready, then you
> should be able to see some errors from the kubelet log.
>
>
> On 3/05/21 8:55 pm, Ammad Syed wrote:
>
> Hi,
>
> I have upgraded my magnum environment from victoria to wallaby. The
> upgrade went successfully. When I am trying to deploy a cluster from
> template, the status of cluster shows UNHEALTHY but create complete.
>
> I have logged into the master nodes and found no error message in heat
> logs. The nodes status still sees NotReady.
>
> [root at k8s-cluster-iomfrpuadezp-master-0 kubernetes]# kubectl get nodes
> --all-namespaces
> NAME                                STATUS     ROLES    AGE     VERSION
> k8s-cluster-iomfrpuadezp-master-0   NotReady   master   14m     v1.18.16
> k8s-cluster-iomfrpuadezp-node-0     NotReady   <none>   9m51s   v1.18.16
>
> Also there is no pods running in kube-system namespace.
>
> [root at k8s-cluster-iomfrpuadezp-master-0 kubernetes]# kubectl get pods
> --all-namespaces
> No resources found
>
> I have checked the logs, the flannel was deployed.
>
> + printf 'Starting to run calico-service\n'
> + set -e
> + set +x
> + '[' flannel = calico ']'
> + printf 'Finished running calico-service\n'
> + set -e
> + set +x
> Finished running calico-service
> + '[' flannel = flannel ']'
> + _prefix=quay.io/coreos/
> + FLANNEL_DEPLOY=/srv/magnum/kubernetes/manifests/flannel-deploy.yaml
> + '[' -f /srv/magnum/kubernetes/manifests/flannel-deploy.yaml ']'
> + echo 'Writing File: /srv/magnum/kubernetes/manifests/flannel-deploy.yaml'
> Writing File: /srv/magnum/kubernetes/manifests/flannel-deploy.yaml
> ++ dirname /srv/magnum/kubernetes/manifests/flannel-deploy.yaml
> + mkdir -p /srv/magnum/kubernetes/manifests
> + set +x
> + '[' '' = 0 ']'
> + /usr/bin/kubectl apply -f
> /srv/magnum/kubernetes/manifests/flannel-deploy.yaml --namespace=kube-system
> podsecuritypolicy.policy/psp.flannel.unprivileged created
> clusterrole.rbac.authorization.k8s.io/flannel created
> clusterrolebinding.rbac.authorization.k8s.io/flannel created
> serviceaccount/flannel created
> configmap/kube-flannel-cfg created
> daemonset.apps/kube-flannel-ds created
>
> I tried to deploy the flannel again, but it showing unchanged.
>
> [root at k8s-cluster-iomfrpuadezp-master-0 heat-config-script]# kubectl
> apply -f /srv/magnum/kubernetes/manifests/flannel-deploy.yaml
> --namespace=kube-system
> podsecuritypolicy.policy/psp.flannel.unprivileged configured
> clusterrole.rbac.authorization.k8s.io/flannel unchanged
> clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
> serviceaccount/flannel unchanged
> configmap/kube-flannel-cfg unchanged
> daemonset.apps/kube-flannel-ds unchanged
>
> The other thing I have noticed that cluster deployment still uses old
> parameters of victoria like heat_agent_tag and others. Its not using latest
> default tags of wallaby release.
>
> I am using magnum on ubuntu 20.04. The other components in stack are
> already upgraded to wallaby release.
>
> --
> Regards,
> Ammad Ali
>
> --
> Cheers & Best regards,
> Feilong Wang (王飞龙)
> ------------------------------------------------------
> Senior Cloud Software Engineer
> Tel: +64-48032246
> Email: flwang at catalyst.net.nz
> Catalyst IT Limited
> Level 6, Catalyst House, 150 Willis Street, Wellington
> ------------------------------------------------------
>
>

-- 
Regards,


Syed Ammad Ali
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210503/eeee501e/attachment-0001.html>