Hi, Michal
I tried it, but it can't work, it is also so even for the network kuryr created by namespace driver, here is some information:
I created namespace by "kubectl create namespace kuryrns1"
yangyi@cmp001:~$ kubectl get ns NAME STATUS AGE default Active 48d kube-node-lease Active 48d kube-public Active 48d kube-system Active 48d kuryrns1 Active 52m
My kuryr conf is below:
yangyi@cmp001:~$ grep "^[^#]" /etc/kuryr/kuryr.conf [DEFAULT] bindir = /home/yangyi/kuryr-k8s-controller/env/libexec/kuryr deployment_type = baremetal log_file = /var/log/kuryr.log [binding] [cache_defaults] [cni_daemon] [cni_health_server] [health_server] [ingress] [kubernetes] api_root = https://10.110.21.64:6443 ssl_client_crt_file = /etc/kubernetes/pki/kuryr.crt ssl_client_key_file = /etc/kubernetes/pki/kuryr.key ssl_ca_crt_file = /etc/kubernetes/pki/ca.crt pod_subnets_driver = namespace enabled_handlers = vif,namespace,kuryrnet [kuryr-kubernetes] [namespace_handler_caching] [namespace_sg] [namespace_subnet] pod_router = 46fc6730-a7f9-45f7-b98b-f682c436e85c pod_subnet_pool = 581daf0e-e661-4fb8-b8d6-b7b11d0b43ab [neutron] auth_url = http://10.110.28.20:35357/v3 auth_type = password password = HAOQNs07Ci9c0DvB project_domain_id = default project_name = admin region_name = SDNRegion tenant_name = admin user_domain_id = default username = admin [neutron_defaults] project = 852d281e70b34b5398c1c5534124952e pod_subnet = b1fa2198-2ecd-41ce-bd06-93ddb2742586 pod_security_groups = d89787f5-b892-487f-b682-88742007f49f ovs_bridge = br-int service_subnet = 58b322fd-19e4-47db-b2fe-5cffd528af05 network_device_mtu = 1450 [node_driver_caching] [np_handler_caching] [octavia_defaults] [pod_ip_caching] [pod_vif_nested] [pool_manager] [sriov] [subnet_caching] [vif_handler_caching] [vif_pool] yangyi@cmp001:~$
KuryrNet has been created automatically by namespace creation:
yangyi@cmp001:~$ kubectl get KuryrNet/ns-kuryrns1 -o yaml apiVersion: openstack.org/v1 kind: KuryrNet metadata: annotations: namespaceName: kuryrns1 creationTimestamp: "2019-10-29T02:58:01Z" generation: 2 name: ns-kuryrns1 resourceVersion: "5926221" selfLink: /apis/openstack.org/v1/kuryrnets/ns-kuryrns1 uid: df5850a5-dc57-4243-b01e-be1c24d788fc spec: netId: 2dcc6969-7923-460e-8ede-17985cdf2b80 populated: true routerId: 46fc6730-a7f9-45f7-b98b-f682c436e85c subnetCIDR: 10.254.0.0/24 subnetId: a46861d3-eccf-4573-8c22-5412cc9d64f0 yangyi@cmp001:~$
But when I created deployment under kuryrns1 namespace, it never succeeded. I found CNI daemon is broken.
It is so before kubectl apply -f deploy.yaml.
yangyi@cmp004:~$ sudo ps aux | grep kuryr root 15339 0.0 0.0 51420 3852 pts/9 S 03:35 0:00 sudo -E kuryr-daemon --config-file /etc/kuryr/kuryr.conf -d root 15340 1.2 0.0 271028 101408 ? Ssl 03:35 0:01 kuryr-daemon: master process [/home/yangyi/kuryr-k8s-cni/env/bin/kuryr-daemon --config-file /etc/kuryr kuryr.conf -d] root 15352 0.0 0.0 268948 92016 ? S 03:35 0:00 kuryr-daemon: master process [/home/yangyi/kuryr-k8s-cni/env/bin/kuryr-daemon --config-file /etc/kuryr kuryr.conf -d] root 15357 0.0 0.0 426944 94624 ? Sl 03:35 0:00 kuryr-daemon: watcher worker(0) root 15362 0.0 0.0 353220 93084 ? Sl 03:35 0:00 kuryr-daemon: server worker(0) root 15366 0.0 0.0 353212 92260 ? Sl 03:35 0:00 kuryr-daemon: health worker(0)
It is so after kubectl apply -f deploy.yaml.
yangyi@cmp004:~$ sudo ps aux | grep kuryr root 15339 0.0 0.0 51420 3852 pts/9 S 03:35 0:00 sudo -E kuryr-daemon --config-file /etc/kuryr/kuryr.conf -d root 15340 0.2 0.0 271028 101408 ? Ssl 03:35 0:01 kuryr-daemon: master process [/home/yangyi/kuryr-k8s-cni/env/bin/kuryr-daemon --config-file /etc/kuryr kuryr.conf -d] root 15352 0.0 0.0 342680 92016 ? Sl 03:35 0:00 kuryr-daemon: master process [/home/yangyi/kuryr-k8s-cni/env/bin/kuryr-daemon --config-file /etc/kuryr kuryr.conf -d] root 15357 0.0 0.0 427200 95028 ? Sl 03:35 0:00 kuryr-daemon: watcher worker(0) root 15362 0.0 0.0 353220 93108 ? Sl 03:35 0:00 kuryr-daemon: server worker(0) root 15366 0.0 0.0 353212 92260 ? Sl 03:35 0:00 kuryr-daemon: health worker(0) root 16426 0.1 0.0 0 0 ? Z 03:39 0:00 [kuryr-daemon: s] <defunct> root 16729 0.0 0.0 428232 94988 ? S 03:40 0:00 kuryr-daemon: server worker(0) root 16813 0.0 0.0 429768 97480 ? S 03:40 0:00 kuryr-daemon: server worker(0) yangyi 17700 0.0 0.0 12944 1012 pts/0 R+ 03:42 0:00 grep --color=auto kuryr
I can see port is indeed created.
yangyi@cmp001:~$ openstack port list --network ns/kuryrns1-net +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ | 2dd5f11f-5fc6-45ee-8f9b-8037019572cd | | fa:16:3e:af:4e:f1 | ip_address='10.254.0.3', subnet_id='a46861d3-eccf-4573-8c22-5412cc9d64f0' | ACTIVE | | c7ff9e5c-1110-4dfa-983d-2f04bf7d2794 | | fa:16:3e:10:64:c7 | ip_address='10.254.0.1', subnet_id='a46861d3-eccf-4573-8c22-5412cc9d64f0' | ACTIVE | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------+--------+ yangyi@cmp001:~$
kuryr log indicated cni is defunct and is restarted.
yangyi@cmp004:~$ grep is_alive /var/log/kuryr.log 2019-10-29 03:40:00.492 16426 DEBUG kuryr_kubernetes.cni.binding.bridge [-] Reporting Driver not healthy. is_alive /home/yangyi/kuryr-k8s-cni/kuryr-kubernetes/kuryr_kubernetes/cni/binding/bridge.py:119 yangyi@cmp004:~$
Can you give me some advice or hints about how I can troubleshoot such an issue?
-----邮件原件----- 发件人: Michał Dulko [mailto:mdulko@redhat.com] 发送时间: 2019年10月22日 23:29 收件人: Yi Yang (杨燚)-云服务集团 yangyi01@inspur.com; ltomasbo@redhat.com 抄送: openstack-discuss@lists.openstack.org 主题: Re: 答复: [kuryr][kuryr-kubernetes] does kuryr-kubernetes support dynamic subnet by pod namespace or annotation?
Oh, I actually should have thought about it. So if you'll precreate the network, subnet and a KuryrNet Custom Resource [1] it should actually work. The definition of KuryrNet can be find here [2], fields are pretty self-explanatory. Please note that you also need to link KuryrNet to the namespace by adding an annotation to the namespace:
"openstack.org/kuryr-net-crd": "ns-<namespace-name>"
Also, just for safety, make sure the KuryrNet itself is named "ns- <namespace-name>" - I'm not sure if some code isn't looking it up by name.
Please note that this was never tested, so maybe there's something I don't see that might prevent it from working.
[1] https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-r... [2] https://github.com/openstack/kuryr-kubernetes/blob/a85a7bc8b1761eb748ccf1643...