<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
At Overstock we do both, in different clouds. Our preferred option is a Ceph backend for Nova ephemeral storage. We like it because it is fast to boot and makes resize easy. Our use case doesn’t require snapshots nor do we have a need for keeping the data
around if a server needs to be rebuilt. It may not work for other people, but it works well for us.
<div class=""><br class="">
</div>
<div class="">In some of our other clouds, where we don’t have Ceph available, we do use Cinder volumes for booting VMs off of backend SAN services. It works ok, but there are a few painpoints in regard to disk resizing - it’s a bit of a cumbersome process
compared the experience with Nova ephemeral. Depending on the solution used, creating the volume for boot can take much much longer and that can be annoying. On the plus side, Cinder does allow you to do QOS to limit I/O, whereas I do not believe that’s
an option with Nova ephemeral. And, again depending on the Cinder solution employed, the disk I/O for this kind of setup can be significantly better than some other options including Nova ephemeral with a Ceph backend. </div>
<div class=""><br class="">
</div>
<div class="">Bottom line: it depends what you need, as both options work well and there are people doing both out there in the wild.</div>
<div class=""><br class="">
</div>
<div class="">Good luck!</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class="">
<div class="">
<div>
<blockquote type="cite" class="">
<div class="">On Aug 1, 2017, at 9:14 AM, John Petrini <<a href="mailto:jpetrini@coredial.com" class="">jpetrini@coredial.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Just my two cents here but we started out using mostly Ephemeral storage in our builds and looking back I wish we hadn't. Note we're using Ceph as a backend so my response is tailored towards Ceph's behavior.
<div class=""><br class="">
</div>
<div class="">The major pain point is snapshots. When you snapshot an nova volume an RBD snapshot occurs and is very quick and uses very little additional storage, however the snapshot is then copied into the images pool and in the process is converted from
a snapshot to a full size image. This takes a long time because you have to copy a lot of data and it takes up a lot of space. It also causes a great deal of IO on the storage and means you end up with a bunch of "snapshot images" creating clutter. On the
other hand volume snapshots are near instantaneous without the other drawbacks I've mentioned.</div>
<div class="gmail_extra"><br class="">
</div>
<div class="gmail_extra">On the plus side for ephemeral storage; resizing the root disk of images works better. As long as your image is configured properly it's just a matter of initiating a resize and letting the instance reboot to grow the root disk. When
using volumes as your root disk you instead have to shutdown the instance, grow the volume and boot.</div>
<div class="gmail_extra"><br class="">
</div>
<div class="gmail_extra">I hope this help! If anyone on the list knows something I don't know regarding these issues please chime in. I'd love to know if there's a better way.</div>
<div class="gmail_extra"><br class="">
</div>
<div class="gmail_extra">Regards,</div>
<div class="gmail_extra">
<div class="">
<div class="gmail_signature" data-smartmail="gmail_signature">
<div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class="">
<p style="background-color:rgb(255,255,255);margin-top:8px;margin-bottom:8px;color:rgb(51,51,51);line-height:1.6" class="">
<span style="font-size:14pt" class="">John Petrini</span></p>
</div>
</div>
<img src="https://t.xink.io/Tracking/Impression/Nx4AAGORAACukSoA0" height="1px" width="1px" style="border: none; display: none !important;" class=""></div>
</div>
<br class="">
<div class="gmail_quote">On Tue, Aug 1, 2017 at 10:50 AM, Kimball, Conrad <span dir="ltr" class="">
<<a href="mailto:conrad.kimball@boeing.com" target="_blank" class="">conrad.kimball@boeing.com</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="#0563C1" vlink="#954F72" class="">
<div class="m_4975858222606055292WordSection1">
<p class="MsoNormal">In our process of standing up an OpenStack internal cloud we are facing the question of ephemeral storage vs. Cinder volumes for instance root disks.<u class=""></u><u class=""></u></p>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal">As I look at public clouds such as AWS and Azure, the norm is to use persistent volumes for the root disk. AWS started out with images booting onto ephemeral disk, but soon after they released Elastic Block Storage and ever since the clear
trend has been to EBS-backed instances, and now when I look at their quick-start list of 33 AMIs, all of them are EBS-backed. And I’m not even sure one can have anything except persistent root disks in Azure VMs.<u class=""></u><u class=""></u></p>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal">Based on this and a number of other factors I think we want our user normal / default behavior to boot onto Cinder-backed volumes instead of onto ephemeral storage. But then I look at OpenStack and its design point appears to be booting
images onto ephemeral storage, and while it is possible to boot an image onto a new volume this is clumsy (haven’t found a way to make this the default behavior) and we are experiencing performance problems (that admittedly we have not yet run to ground).<u class=""></u><u class=""></u></p>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal">So …<u class=""></u><u class=""></u></p>
<p class="m_4975858222606055292MsoListParagraph" style="margin-left:.25in"><u class=""></u><span style="font-family:Symbol" class=""><span class="">·<span style="font:7.0pt "Times New Roman"" class="">
</span></span></span><u class=""></u>Are other operators routinely booting onto Cinder volumes instead of ephemeral storage?<u class=""></u><u class=""></u></p>
<p class="m_4975858222606055292MsoListParagraph" style="margin-left:.25in"><u class=""></u><span style="font-family:Symbol" class=""><span class="">·<span style="font:7.0pt "Times New Roman"" class="">
</span></span></span><u class=""></u>What has been your experience with this; any advice?<u class=""></u><u class=""></u></p>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
<p class="MsoNormal"><i class=""><span style="font-size:12.0pt;color:#1f497d" class="">Conrad Kimball<u class=""></u><u class=""></u></span></i></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">Associate Technical Fellow<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">Chief Architect, Enterprise Cloud Services<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">Application Infrastructure Services / Global IT Infrastructure / Information Technology & Data Analytics<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class=""><a href="mailto:conrad.kimball@boeing.com" target="_blank" class=""><span style="color:#0563c1" class="">conrad.kimball@boeing.com</span></a><u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">P.O. Box 3707, Mail Code 7M-TE<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">Seattle, WA 98124-2207<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">Bellevue 33-11 bldg, office 3A6-3.9<u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1f497d" class="">Mobile:
<a href="tel:(425)%20591-7802" value="+14255917802" target="_blank" class="">425-591-7802</a><u class=""></u><u class=""></u></span></p>
<p class="MsoNormal"><u class=""></u> <u class=""></u></p>
</div>
</div>
<br class="">
______________________________<wbr class="">_________________<br class="">
OpenStack-operators mailing list<br class="">
<a href="mailto:OpenStack-operators@lists.openstack.org" class="">OpenStack-operators@lists.<wbr class="">openstack.org</a><br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank" class="">http://lists.openstack.org/<wbr class="">cgi-bin/mailman/listinfo/<wbr class="">openstack-operators</a><br class="">
<br class="">
</blockquote>
</div>
<br class="">
</div>
</div>
_______________________________________________<br class="">
OpenStack-operators mailing list<br class="">
<a href="mailto:OpenStack-operators@lists.openstack.org" class="">OpenStack-operators@lists.openstack.org</a><br class="">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or
the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in
error, please notify sender immediately by telephone or return email. Thank you.<br>
</font>
</body>
</html>