[openstack-dev] [Magnum]How do we support swarm master node HA
Qiao,Liyong
liyong.qiao at intel.com
Tue Nov 24 11:46:43 UTC 2015
hi all,
failed to set up swarm master HA cluster:
here is my setup .
2 master and 1 node, and I put a LB in front of 2 master node(Round-Robin)
I saw from swarm ha guide and get confirmed from swarm guys:
/jimmyxian | elqiao: I'm here. :). Swarm does not support A-A. But can
access the standby manager, and it will proxy the request to the primary
manager/
https://docs.docker.com/swarm/multi-manager-setup/
Swarm replica will do the proxy. but I tested failed.
Since the LB use Round-Robin mode so it will access primary then replica.
every time if LB access primary node, the cluster works fine, but failed
when access replica.
I wonder if the configuration is wrong ?
/here is the ENV detail:/
master-1 172.24.5.33(floating ip) 192.168.0.5:2(private ip) primary
root 1289 0.1 1.4 35456 29272 ? Ssl 10:02 0:07 /swarm
manage -H tcp://0.0.0.0:2375 --replication --advertise 192.168.0.5:2375
--tlsverify --tlscacert=/etc/docker/ca.crt
--tlskey=/etc/docker/server.key --tlscert=/etc/docker/server.crt
etcd://192.168.0.3:2379/v2/keys/swarm/
master-2 172.24.5.32(floating ip) 192.168.0.6(private ip) replica
root 1678 0.1 0.8 23572 16824 ? Ssl 11:31 0:00 /swarm
manage -H tcp://0.0.0.0:2375 --replication --advertise 192.168.0.6:2375
--tlsverify --tlscacert=/etc/docker/ca.crt
--tlskey=/etc/docker/server.key --tlscert=/etc/docker/server.crt
etcd://192.168.0.3:2379/v2/keys/swarm/
on master-1 172.24.5.33 (primary)
bash-4.3# docker -H tcp://172.24.5.33:2376 --tlsverify --tlscacert
ca.crt --tlskey server.key --tlscert server.crt info
Containers: 6
Images: 6
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 3
sw-6ckizfpu4bl-0-mjy7qmxwbc6s-swarm-node-bynksfbxgibf.novalocal:
192.168.0.7:2375
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2,
kernelversion=3.17.4-301.fc21.x86_64, operatingsystem=Fedora 21 (Twenty
One), storagedriver=devicemapper
sw-ivtl4icqr-0-7a7s2ycpss2k-swarm-master-mxihlwsyjetc.novalocal:
192.168.0.5:2375
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2,
kernelversion=3.17.4-301.fc21.x86_64, operatingsystem=Fedora 21 (Twenty
One), storagedriver=devicemapper
sw-ivtl4icqr-1-35oewlqh25a7-swarm-master-idtxokrzgaek.novalocal:
192.168.0.6:2375
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2,
kernelversion=3.17.4-301.fc21.x86_64, operatingsystem=Fedora 21 (Twenty
One), storagedriver=devicemapper
CPUs: 3
Total Memory: 6.158 GiB
Name: 78443d1d9ad2
Http Proxy: http://10.239.4.160:911/
Https Proxy: https://10.239.4.160:911/
No Proxy:
192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4,192.168.0.5,192.168.0.6,192.168.0.7
I can see all containers of the cluster:
bash-4.3# docker -H tcp://172.24.5.32:2376 --tlsverify --tlscacert
ca.crt --tlskey server.key --tlscert server.crt ps -a
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
78443d1d9ad2 swarm:1.0.0 "/swarm manage -H tcp" About an
hour ago Up About an hour 0.0.0.0:2376->2375/tcp swarm-manager
d19e9ab13e07 swarm:1.0.0 "/swarm join --addr 1" About an
hour ago Up About an hour 2375/tcp swarm-agent
bash-4.3# docker -H tcp://172.24.5.33:2376 --tlsverify --tlscacert
ca.crt --tlskey server.key --tlscert server.crt ps -a
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
0337ad1ad6a6 docker.io/cirros "ping -c 100 10.248.2" 50
minutes ago Exited (137) 26 minutes ago
sw-6ckizfpu4bl-0-mjy7qmxwbc6s-swarm-node-bynksfbxgibf.novalocal/test_ping
6a6e1f1327e2 swarm:1.0.0 "/swarm join --addr 1" About an
hour ago Up About an hour 2375/tcp
sw-6ckizfpu4bl-0-mjy7qmxwbc6s-swarm-node-bynksfbxgibf.novalocal/swarm-agent
78443d1d9ad2 swarm:1.0.0 "/swarm manage -H tcp" About an
hour ago Up About an hour 192.168.0.5:2376->2375/tcp
sw-ivtl4icqr-0-7a7s2ycpss2k-swarm-master-mxihlwsyjetc.novalocal/swarm-manager
d19e9ab13e07 swarm:1.0.0 "/swarm join --addr 1" About an
hour ago Up About an hour 2375/tcp
sw-ivtl4icqr-0-7a7s2ycpss2k-swarm-master-mxihlwsyjetc.novalocal/swarm-agent
a4da371274bc swarm:1.0.0 "/swarm manage -H tcp" About an
hour ago Up 3 minutes 192.168.0.6:2376->2375/tcp
sw-ivtl4icqr-1-35oewlqh25a7-swarm-master-idtxokrzgaek.novalocal/swarm-manager
a211d31dfc6e swarm:1.0.0 "/swarm join --addr 1" About an
hour ago Up About an hour 2375/tcp
sw-ivtl4icqr-1-35oewlqh25a7-swarm-master-idtxokrzgaek.novalocal/swarm-agent
=========================================================================
on master-2 172.24.5.32(replica)
bash-4.3# docker -H tcp://172.24.5.32:2376 --tlsverify --tlscacert
ca.crt --tlskey server.key --tlscert server.crt info
Containers: 6
Images: 6
Role: replica
Primary: 192.168.0.5:2375
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 3
sw-6ckizfpu4bl-0-mjy7qmxwbc6s-swarm-node-bynksfbxgibf.novalocal:
192.168.0.7:2375
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2,
kernelversion=3.17.4-301.fc21.x86_64, operatingsystem=Fedora 21 (Twenty
One), storagedriver=devicemapper
sw-ivtl4icqr-0-7a7s2ycpss2k-swarm-master-mxihlwsyjetc.novalocal:
192.168.0.5:2375
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2,
kernelversion=3.17.4-301.fc21.x86_64, operatingsystem=Fedora 21 (Twenty
One), storagedriver=devicemapper
sw-ivtl4icqr-1-35oewlqh25a7-swarm-master-idtxokrzgaek.novalocal:
192.168.0.6:2375
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=native-0.2,
kernelversion=3.17.4-301.fc21.x86_64, operatingsystem=Fedora 21 (Twenty
One), storagedriver=devicemapper
CPUs: 3
Total Memory: 6.158 GiB
Name: a4da371274bc
Http Proxy: http://10.239.4.160:911/
Https Proxy: https://10.239.4.160:911/
No Proxy:
192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4,192.168.0.5,192.168.0.6,192.168.0.7
*Only local containers can be list:*
bash-4.3# docker -H tcp://172.24.5.32:2376 --tlsverify --tlscacert
ca.crt --tlskey server.key --tlscert server.crt ps -a
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
78443d1d9ad2 swarm:1.0.0 "/swarm manage -H tcp" About an
hour ago Up About an hour 0.0.0.0:2376->2375/tcp swarm-manager
d19e9ab13e07 swarm:1.0.0 "/swarm join --addr 1" About an
hour ago Up About an hour 2375/tcp swarm-agent
--
BR, Eli(Li Yong)Qiao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151124/cc413bfd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: liyong_qiao.vcf
Type: text/x-vcard
Size: 123 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151124/cc413bfd/attachment.vcf>
More information about the OpenStack-dev
mailing list