<div dir="ltr">Hi Hongbin,<div><br></div><div><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">The docker in master node stores data in /dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--data and metadata in /dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--meta. </span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">/dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--data and </span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">/dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--meta are logic volumes. The docker in minion node store data in the cinder volume, but </span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">/dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--meta and </span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">/dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--meta are not used. If we want to leverage Cinder volume for docker in master, should we drop </span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">/dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--meta and </span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">/dev/mapper/atomicos-</span><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px">docker--meta? I think it is not necessary to allocate a Cinder volume. It is enough to allocate two logic volumes for docker, because only etcd, flannel, k8s run in the docker daemon which need not a large amount of storage.</span><br></div><div><span style="color:rgb(80,0,80);font-family:Calibri,sans-serif;font-size:14px"><br></span></div><div><font color="#500050" face="Calibri, sans-serif"><span style="font-size:14px">Best regards,</span></font></div><div><font color="#500050" face="Calibri, sans-serif"><span style="font-size:14px">Wanghua</span></font></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 26, 2015 at 12:40 AM, Hongbin Lu <span dir="ltr"><<a href="mailto:hongbin.lu@huawei.com" target="_blank">hongbin.lu@huawei.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-CA" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Here is a bit more context.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Currently, at k8s and swarm bay, some required binaries (i.e. etcd and flannel) are built into image and run at host. We are exploring the possibility to containerize
some of these system components. The rationales are (i) it is infeasible to build custom packages into an atomic image and (ii) it is infeasible to upgrade individual component. For example, if there is a bug in current version of flannel and we know the bug
was fixed in the next version, we need to upgrade flannel by building a new image, which is a tedious process.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">To containerize flannel, we need a second docker daemon, called docker-bootstrap [1]. In this setup, pods are running on the main docker daemon, and flannel
and etcd are running on the second docker daemon. The reason is that flannel needs to manage the network of the main docker daemon, so it needs to run on a separated daemon.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Daneyon, I think it requires separated storage because it needs to run a separated docker daemon (unless there is a way to make two docker daemons share the
same storage).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Wanghua, is it possible to leverage Cinder volume for that. Leveraging external storage is always preferred [2].<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[1] <a href="http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html#bootstrap-docker" target="_blank">http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html#bootstrap-docker</a></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2]
<a href="http://www.projectatomic.io/docs/docker-storage-recommendation/" target="_blank">http://www.projectatomic.io/docs/docker-storage-recommendation/</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Best regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hongbin<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Daneyon Hansen (danehans) [mailto:<a href="mailto:danehans@cisco.com" target="_blank">danehans@cisco.com</a>]
<br>
<b>Sent:</b> November-25-15 11:10 AM<span class=""><br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
</span><b>Subject:</b> Re: [openstack-dev] [magnum]storage for docker-bootstrap<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:
</span></b><span lang="ZH-CN" style="font-size:11.0pt;color:black">王华</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> <<a href="mailto:wanghua.humble@gmail.com" target="_blank">wanghua.humble@gmail.com</a>><br>
<b>Reply-To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Date: </b>Wednesday, November 25, 2015 at 5:00 AM<br>
<b>To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<b>Subject: </b>[openstack-dev] [magnum]storage for docker-bootstrap<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">Hi all,
<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">I am working on containerizing etcd and flannel. But I met a problem. As described in [1], we need a docker-bootstrap. Docker and docker-bootstrap can not use
the same storage, so we need some disk space for it.<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">I reviewed [1] and I do not see where the bootstrap docker instance requires separate storage.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<blockquote style="border:none;border-left:solid #b5c4df 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-right:0cm">
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">The docker in master node stores data in /dev/mapper/atomicos-docker--data and metadata in /dev/mapper/atomicos-docker--meta. The disk space left is too same
for docker-bootstrap. Even if the root_gb of the instance flavor is 20G, only 8G can be used in our image. I want to make it bigger. One way is we can add the disk space left in the vda as vda3 into atomicos vg after the instance starts and we allocate two
logic volumes for docker-bootstrap. Another way is when we create the image, we allocate two logic volumes for docker-bootstrap. The second way has a advantage. It doesn't have to make filesystem when the instance is created which is time consuming.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">What is your opinion?<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">Best Regards<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">Wanghua<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Calibri","sans-serif";color:black">[1] <a href="http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode/master.html" target="_blank">http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode/master.html</a><u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</blockquote>
</div></div></div>
</div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>