Open Stack

Hi all,

If we want to run etcd and flannel in container, we will introduce
docker-bootstrap
which makes setup become more complex as Egor pointed out. Should we pay
for the price?

On Sat, Nov 28, 2015 at 8:45 AM, Egor Guz <EGuz at walmartlabs.com> wrote:

> Wanghua,
>
> I don’t think moving flannel to the container is good idea. This is setup
> great for dev environment, but become too complex from operator point of
> view (you add extra Docker daemon and need extra Cinder volume for this
> daemon, also
> keep in mind it makes sense to keep etcd data folder at Cinder storage as
> well because etcd is database). flannel has just there files without extra
> dependencies and it’s much easy to download it during cloud-init ;)
>
> I agree that we have pain with building Fedora Atomic images, but instead
> of simplify this process we should switch to another more “friendly” images
> (e.g. Fedora/CentOS/Ubuntu) which we can easy build with disk builder.
> Also we can fix CoreOS template (I believe people more asked about it
> instead of Atomic), but we may face similar to Atomic issues when we will
> try to integrate not CoreOS products (e.g. Calico or Weave)
>
> —
> Egor
>
> From: 王华 <wanghua.humble at gmail.com<mailto:wanghua.humble at gmail.com>>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org
> >>
> Date: Thursday, November 26, 2015 at 00:15
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org
> >>
> Subject: Re: [openstack-dev] [magnum]storage for docker-bootstrap
>
> Hi Hongbin,
>
> The docker in master node stores data in /dev/mapper/atomicos-docker--data
> and metadata in /dev/mapper/atomicos-docker--meta.
> /dev/mapper/atomicos-docker--data and /dev/mapper/atomicos-docker--meta are
> logic volumes. The docker in minion node store data in the cinder volume,
> but /dev/mapper/atomicos-docker--meta and /dev/mapper/atomicos-docker--meta
> are not used. If we want to leverage Cinder volume for docker in master,
> should we drop /dev/mapper/atomicos-docker--meta and
> /dev/mapper/atomicos-docker--meta? I think it is not necessary to allocate
> a Cinder volume. It is enough to allocate two logic volumes for docker,
> because only etcd, flannel, k8s run in the docker daemon which need not a
> large amount of storage.
>
> Best regards,
> Wanghua
>
> On Thu, Nov 26, 2015 at 12:40 AM, Hongbin Lu <hongbin.lu at huawei.com
> <mailto:hongbin.lu at huawei.com>> wrote:
> Here is a bit more context.
>
> Currently, at k8s and swarm bay, some required binaries (i.e. etcd and
> flannel) are built into image and run at host. We are exploring the
> possibility to containerize some of these system components. The rationales
> are (i) it is infeasible to build custom packages into an atomic image and
> (ii) it is infeasible to upgrade individual component. For example, if
> there is a bug in current version of flannel and we know the bug was fixed
> in the next version, we need to upgrade flannel by building a new image,
> which is a tedious process.
>
> To containerize flannel, we need a second docker daemon, called
> docker-bootstrap [1]. In this setup, pods are running on the main docker
> daemon, and flannel and etcd are running on the second docker daemon. The
> reason is that flannel needs to manage the network of the main docker
> daemon, so it needs to run on a separated daemon.
>
> Daneyon, I think it requires separated storage because it needs to run a
> separated docker daemon (unless there is a way to make two docker daemons
> share the same storage).
>
> Wanghua, is it possible to leverage Cinder volume for that. Leveraging
> external storage is always preferred [2].
>
> [1]
> http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html#bootstrap-docker
> [2] http://www.projectatomic.io/docs/docker-storage-recommendation/
>
> Best regards,
> Hongbin
>
> From: Daneyon Hansen (danehans) [mailto:danehans at cisco.com<mailto:
> danehans at cisco.com>]
> Sent: November-25-15 11:10 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [magnum]storage for docker-bootstrap
>
>
>
> From: 王华 <wanghua.humble at gmail.com<mailto:wanghua.humble at gmail.com>>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org
> >>
> Date: Wednesday, November 25, 2015 at 5:00 AM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org
> >>
> Subject: [openstack-dev] [magnum]storage for docker-bootstrap
>
> Hi all,
>
> I am working on containerizing etcd and flannel. But I met a problem. As
> described in [1], we need a docker-bootstrap. Docker and docker-bootstrap
> can not use the same storage, so we need some disk space for it.
>
> I reviewed [1] and I do not see where the bootstrap docker instance
> requires separate storage.
>
> The docker in master node stores data in /dev/mapper/atomicos-docker--data
> and metadata in /dev/mapper/atomicos-docker--meta. The disk space left is
> too same for docker-bootstrap. Even if the root_gb of the instance flavor
> is 20G, only 8G can be used in our image. I want to make it bigger. One way
> is we can add the disk space left in the vda as vda3 into atomicos vg after
> the instance starts and we allocate two logic volumes for docker-bootstrap.
> Another way is when we create the image, we allocate two logic volumes for
> docker-bootstrap. The second way has a advantage. It doesn't have to make
> filesystem when the instance is created which is time consuming.
>
> What is your opinion?
>
> Best Regards
> Wanghua
>
> [1]
> http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode/master.html
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151207/7a0c73ba/attachment.html>