[kolla-ansible] Running Docker Swarm with Kuryr networking
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce https://docs.openstack.org/kolla-ansible/latest/reference/containers/kuryr-guide.html and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi, long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case. Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance, Paolo
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce https://docs.openstack.org/kolla-ansible/latest/reference/containers/kuryr-guide.html and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On Wed, Sep 1, 2021 at 6:08 PM Paolo Celati paolo@celati.com wrote:
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
We actually *do* test Kuryr in CI - in the Zun scenario - and it works, at least on CentOS, it fails on Ubuntu for some reason but we don't have anyone to take care of that... Zun works using the on-host Docker as you want to do. *But* as Michał Dulko (hah, had to add the surname to differentiate ;-) ) said non-Kubernetes Kuryr is largely unmaintained and, actually, a similar statement applies to Zun... Your mileage may vary but going forward I advise you take the time to learn Kubernetes. As for mixing up Docker Swarm with Kolla Ansible, that's not really supported either. Kolla Ansible is designed to deploy to a bunch of standalone Docker daemons using only host networking. It could work but it's not tested.
-yoctozepto
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On Wed, 2021-09-01 at 19:28 +0200, Radosław Piliszek wrote:
On Wed, Sep 1, 2021 at 6:08 PM Paolo Celati paolo@celati.com wrote:
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
We actually *do* test Kuryr in CI - in the Zun scenario - and it works, at least on CentOS, it fails on Ubuntu for some reason but we don't have anyone to take care of that... Zun works using the on-host Docker as you want to do.
I had a feeling that Zun is using kuryr-libnetwork, which is a different thing than kuryr itself. If the question if about kuryr- libnetwork, then it's maintained better and Hongbin Lu is an expert to try contacting with. Ping us on #openstack-kuryr and we'll try to get you his whereabouts.
*But* as Michał Dulko (hah, had to add the surname to differentiate ;-) ) said non-Kubernetes Kuryr is largely unmaintained and, actually, a similar statement applies to Zun... Your mileage may vary but going forward I advise you take the time to learn Kubernetes. As for mixing up Docker Swarm with Kolla Ansible, that's not really supported either. Kolla Ansible is designed to deploy to a bunch of standalone Docker daemons using only host networking. It could work but it's not tested.
-yoctozepto
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On Thu, Sep 2, 2021 at 10:21 AM Michał Dulko mdulko@redhat.com wrote:
On Wed, 2021-09-01 at 19:28 +0200, Radosław Piliszek wrote:
On Wed, Sep 1, 2021 at 6:08 PM Paolo Celati paolo@celati.com wrote:
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
We actually *do* test Kuryr in CI - in the Zun scenario - and it works, at least on CentOS, it fails on Ubuntu for some reason but we don't have anyone to take care of that... Zun works using the on-host Docker as you want to do.
I had a feeling that Zun is using kuryr-libnetwork, which is a different thing than kuryr itself. If the question if about kuryr- libnetwork, then it's maintained better and Hongbin Lu is an expert to try contacting with. Ping us on #openstack-kuryr and we'll try to get you his whereabouts.
Ah, yeah. In Kolla we say "kuryr" but mean "kuryr-libnetwork". We actually install both kuryr and kuryr-libnetwork in the container and run kuryr-server. Perhaps mistakenly? I see you also have kuryr-lib for common code. The Kuryr integration was contributed by none of the current Kolla folks so we might not understand the relationships well... We will gladly accept improvements there.
-yoctozepto
*But* as Michał Dulko (hah, had to add the surname to differentiate ;-) ) said non-Kubernetes Kuryr is largely unmaintained and, actually, a similar statement applies to Zun... Your mileage may vary but going forward I advise you take the time to learn Kubernetes. As for mixing up Docker Swarm with Kolla Ansible, that's not really supported either. Kolla Ansible is designed to deploy to a bunch of standalone Docker daemons using only host networking. It could work but it's not tested.
-yoctozepto
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On Thu, 2021-09-02 at 10:39 +0200, Radosław Piliszek wrote:
On Thu, Sep 2, 2021 at 10:21 AM Michał Dulko mdulko@redhat.com wrote:
On Wed, 2021-09-01 at 19:28 +0200, Radosław Piliszek wrote:
On Wed, Sep 1, 2021 at 6:08 PM Paolo Celati paolo@celati.com wrote:
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
We actually *do* test Kuryr in CI - in the Zun scenario - and it works, at least on CentOS, it fails on Ubuntu for some reason but we don't have anyone to take care of that... Zun works using the on-host Docker as you want to do.
I had a feeling that Zun is using kuryr-libnetwork, which is a different thing than kuryr itself. If the question if about kuryr- libnetwork, then it's maintained better and Hongbin Lu is an expert to try contacting with. Ping us on #openstack-kuryr and we'll try to get you his whereabouts.
Ah, yeah. In Kolla we say "kuryr" but mean "kuryr-libnetwork". We actually install both kuryr and kuryr-libnetwork in the container and run kuryr-server. Perhaps mistakenly? I see you also have kuryr-lib for common code. The Kuryr integration was contributed by none of the current Kolla folks so we might not understand the relationships well... We will gladly accept improvements there.
Yeah, it took me a while to get the relationships. :/ Here's how it works:
* openstack/kuryr - Docker legacy network plugin *and* kuryr-lib source. * openstack/kuryr-libnetwork - Docker libnetwork implementation. * openstack/kuryr-kubernetes - (CNCF) CNI implementation, to be used in K8s.
-yoctozepto
*But* as Michał Dulko (hah, had to add the surname to differentiate ;-) ) said non-Kubernetes Kuryr is largely unmaintained and, actually, a similar statement applies to Zun... Your mileage may vary but going forward I advise you take the time to learn Kubernetes. As for mixing up Docker Swarm with Kolla Ansible, that's not really supported either. Kolla Ansible is designed to deploy to a bunch of standalone Docker daemons using only host networking. It could work but it's not tested.
-yoctozepto
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On Thu, Sep 2, 2021 at 10:58 AM Michał Dulko mdulko@redhat.com wrote:
On Thu, 2021-09-02 at 10:39 +0200, Radosław Piliszek wrote:
On Thu, Sep 2, 2021 at 10:21 AM Michał Dulko mdulko@redhat.com wrote:
On Wed, 2021-09-01 at 19:28 +0200, Radosław Piliszek wrote:
On Wed, Sep 1, 2021 at 6:08 PM Paolo Celati paolo@celati.com wrote:
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
We actually *do* test Kuryr in CI - in the Zun scenario - and it works, at least on CentOS, it fails on Ubuntu for some reason but we don't have anyone to take care of that... Zun works using the on-host Docker as you want to do.
I had a feeling that Zun is using kuryr-libnetwork, which is a different thing than kuryr itself. If the question if about kuryr- libnetwork, then it's maintained better and Hongbin Lu is an expert to try contacting with. Ping us on #openstack-kuryr and we'll try to get you his whereabouts.
Ah, yeah. In Kolla we say "kuryr" but mean "kuryr-libnetwork". We actually install both kuryr and kuryr-libnetwork in the container and run kuryr-server. Perhaps mistakenly? I see you also have kuryr-lib for common code. The Kuryr integration was contributed by none of the current Kolla folks so we might not understand the relationships well... We will gladly accept improvements there.
Yeah, it took me a while to get the relationships. :/ Here's how it works:
- openstack/kuryr - Docker legacy network plugin *and* kuryr-lib source.
- openstack/kuryr-libnetwork - Docker libnetwork implementation.
- openstack/kuryr-kubernetes - (CNCF) CNI implementation, to be used in K8s.
Thank you!
It seems we are doing the right thing then. ;-)
-yoctozepto
-yoctozepto
*But* as Michał Dulko (hah, had to add the surname to differentiate ;-) ) said non-Kubernetes Kuryr is largely unmaintained and, actually, a similar statement applies to Zun... Your mileage may vary but going forward I advise you take the time to learn Kubernetes. As for mixing up Docker Swarm with Kolla Ansible, that's not really supported either. Kolla Ansible is designed to deploy to a bunch of standalone Docker daemons using only host networking. It could work but it's not tested.
-yoctozepto
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On 01/09/2021 19:28, Radosław Piliszek wrote:
On Wed, Sep 1, 2021 at 6:08 PM Paolo Celati paolo@celati.com wrote:
Hi Michal,
Yes I would rather run all my Docker containers on the physical hosts. At the moment I run Ceph and kolla images, but adding Swarm as well would be useful.
I've had a fair number of problems with Magnum because it appears only Kubernetes is supported nowadays, and I have 0 experience with that. I also prefer Swarm because it's a lot simpler for small scale.
Administering my own VMs with Docker Swarm on top is also not exactly the best solution because that introduces extra machines to maintain. And I don't have lots of spare RAM either as it's a homelab.
Thanks for reminding me about tagging correctly, I forgot to put [kuryr]. Ok didn't know about the lack of CI but good to know.
We actually *do* test Kuryr in CI - in the Zun scenario - and it works, at least on CentOS, it fails on Ubuntu for some reason but we don't have anyone to take care of that... Zun works using the on-host Docker as you want to do. *But* as Michał Dulko (hah, had to add the surname to differentiate ;-) ) said non-Kubernetes Kuryr is largely unmaintained and, actually, a similar statement applies to Zun... Your mileage may vary but going forward I advise you take the time to learn Kubernetes. As for mixing up Docker Swarm with Kolla Ansible, that's not really supported either. Kolla Ansible is designed to deploy to a bunch of standalone Docker daemons using only host networking. It could work but it's not tested.
-yoctozepto
Hi Radoslaw, Ok thanks I'll look into Kubernetes then, it looks like everything is going in that direction anyways so might as well. Well what I was intending to do with Docker Swarm is really just share the hosts running Kolla containers and Swarm containers. It makes for a lot less management work to have fewer Docker daemons. My idea was to have Kolla containers use host networking and Swarm uses overlay (vxlan) networks via Kuryr. But I'll probably try getting Magnum + Kubernetes up instead since that seems the best supported scenario.
Paolo
Kind regards,
Paolo
On 01/09/2021 15:57, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla-Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi,
long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case.
Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster-advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance,
Paolo
On Wed, 2021-09-01 at 15:57 +0200, Michał Nasiadka wrote:
Hi Paolo,
Would you like to use the Docker engine that is running on the OpenStack cluster hosts, or create Virtual Machines that will be used for a Docker Swarm cluster? I would propose the latter.
About Kuryr - we don’t have CI coverage for testing Kuryr in Kolla- Ansible deployment, so the container images and Ansible deployment role are provided as-is currently.
Maybe somebody from Kuryr project could help you out? Adding [kuryr] tag for visibility.
I doubt anybody on the current team has any experience with running Docker's Kuryr. We're focused on kuryr-kubernetes and Kuryr is in de- facto unmaintained state.
Best regards,
Michal On 1 Sep 2021, 00:32 +0200, Paolo Celati paolo@celati.com, wrote:
Hi, long story short I have a 3 node Openstack cluster that I manage with kolla-ansible, and I'd like to run Docker Swarm on that as well. I am aware Magnum exists, but I'd first like to get my head around this simpler case. Seeing as I'd like to connect Docker containers from swarm compose files to Neutron networks I'm trying to set up Kuryr together with a swarm configuration. However the documentation is a little scarce and I'd prefer running everything on these three hosts, including etcd. If I follow the guide and pass --cluster-store and --cluster- advertise arguments to dockerd then I can't run Docker in Swarm mode because I get an error saying Swarm is incompatible with those options, and at the same time it's not clear from documentation how you are expected to do Kuryr+Swarm. I did initialise the Swarm cluster before trying to add Kuryr, so I don't know if perhaps doing this the other way works? Do you have ideas or advice with this scenario? If worst comes to worst I can set up an external etcd cluster on a separate non-Openstack cluster but I'd rather avoid that.
Thanks in advance, Paolo
participants (4)
-
Michał Dulko
-
Michał Nasiadka
-
Paolo Celati
-
Radosław Piliszek