<div dir="ltr"><div>Thanks for posting this John. I'm not good about looking through the dev lists, so it's much appreciated here. <br><br>Regarding your query, I'm having a hard time envisioning good use cases for this. I have a few major issues with this. The primary use case would be for something like a database or key-value store, but that is better handled by the application itself. I would like to think that it isn't a bunch of volumes with static files. Also, I think this is much larger than Cinder itself. We're probably talking about either AZ or region failover, in which case, this is a lot of wasted resources at any sort of appreciable scale, if it's active/passive, and presents a myriad of problems when dealing with actual failover.<br>
<br></div><div>I understand that there are storage vendors that offer site replication, and that exposing those through Cinder would provide value. That could still be accomplished through the types, as you mentioned, and I think that's probably better, at least for now. I'm open to use cases where this would be valuable, but I suspect those same ones might be better served by looking hard at the application engineering.<br>
</div><div><br></div>FWIW, I work @ Comcast. We'll have some representation at the San Antonio Ops meetup, if you're going.<br><div><div class="gmail_extra"><br clear="all"><div>Warren Wang<br></div>
<br><br><div class="gmail_quote">On Sun, Jul 27, 2014 at 11:32 AM, John Griffith <span dir="ltr"><<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div class="h5"><div style="font-family:courier new,monospace"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Jul 26, 2014 at 12:58 PM, Tim Bell <span dir="ltr"><<a href="mailto:Tim.Bell@cern.ch" target="_blank">Tim.Bell@cern.ch</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="#0563C1" vlink="#954F72" lang="EN-GB">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The terms do vary significantly between different vendors which adds to the confusion. I’ll try to add what we see from a CERN perspective
(which is not always your typical customer profile but we use ceph and NetApp). I assume this would still follow the approach where the storage subsystem is in charge and cinder issues the commands to set things up.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Replication for us would mean copy of a volume to a remote location. Parameterisation such as copy type (async/sync), maximum delay
and target pool would be expected.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Consistency groups for me would be how to handle multiple volume replication in terms of sync points to ensure a consistent set of
volumes. Allowing a disaster recovery scenario is our most obvious use case, i.e. restart application VMs remotely.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">We have use cases for replication (such as online database logs), less for consistency groups but as we move more into VMs with external
system disk/data disk models where we’d like the two to be in sync if possible.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">What are the specific questions that you have in mind ?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Tim<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"" lang="EN-US">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"" lang="EN-US"> John Griffith [mailto:<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>]
<br>
<b>Sent:</b> 26 July 2014 16:29<br>
<b>To:</b> <a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a><br>
<b>Subject:</b> [Openstack-operators] [OpenStack-Operators] [Cinder] Request for input on new/advanced features<u></u><u></u></span></p>
</div>
</div><div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">1. Replication<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">2. Consistency Groups<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">It's easy for Vendors to come to me and say "we have lots of customers asking for this" but personally I'd love to get feedback directly from the actual customers, that's where you all come in.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">So, if any of you out there are interested in these topics from a Cinder perspective I'd love to hear from you. If there's interest we can either start an ML thread to discuss, or perhaps a meeting
to catch everybody up and hash things out a bit. I'd like to discuss the current proposals and go through what you as Operators may feel should be priorities (or better yet things you don't care about). I make no promises on the outcome of this little experiment
but thought it would be interesting to try and get user input up front before we release new features.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Let me know your thoughts, I'll avoid a bunch of detail in this posting until I get a feel for who if anybody is interested in helping out.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Thanks,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">John<u></u><u></u></span></p>
</div>
</div>
</div></div>
</div>
</div>
</blockquote></div><br></div></div></div><div class="gmail_extra"><div style="font-family:'courier new',monospace">Hey Tim,</div><div style="font-family:'courier new',monospace">
<br></div><div style="font-family:'courier new',monospace">Thanks for the response and input. The details of my email were a bit vague by intention, but maybe it would be better if I gave some more background. At least on the replication topic. First though, one thing I'd like to do is get general input from Operator Community on features in Cinder (ie; are we working on the right things, are we completely missing key features you need or want).</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">Replication is an interesting one and there are a number of ways to go in terms of how to implement it in Cinder. Here are some of the paths I see:</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">1. Implement new API methods specifically to manage replication.</div>
<div style="font-family:'courier new',monospace">This includes providing API commands for a replicated volume object including create, enable, disable, delete and promote. In essence this adds a new managed object to Cinder.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">2. Leave the replication implementation and control completely up to the driver and expose it via Volume-Type and extra-specs info. The driver would create setup and create the replication target based on the Type input. In this case the replication target is probably invisible to the end-user, although we could add this sort of info to the status update info from the driver.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">Primary use case for both of these approaches is providing the ability to have volumes that are replicated on remote devices (remote sites or otherwise) to use in a DR scenario. There has been some talk about mirrored regional deployments which I think could be obtained by either of these approaches but currently that's not the primary focus for this first release, that would be focused work on a follow up release in my opinion.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">So my questions were of course is this a huge gap for Cinder to begin with, and are their details in the use case and how this should be consumed that we might be missing.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">One of the difficult things about approach '1' listed above is that we'll have a period of time in which most drivers won't actually support the commands. That's not really a terribly big deal but it does add some confusion. What I'm more concerned about is the complexity it introduces to every volume action we do in Cinder. Again, nothing we can't work with and modify but it's worth considering I think.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">Approach '2' is somewhat on the other end of the spectrum. It's very basic, doesn't offer a ton of value add, but it provides some good first steps to enabling replication. It also eliminates some of the concerns I have regarding API methods being in Cinder that aren't going to work with most of the drivers (granted this would be addressed over time).</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">Anyway, I think there's a middle-ground between the two approaches we have in progress right now. The big question I had for the community however is if this is something that over time the Operator Community has been feeling like "oh if we just had replication", or "if we just had feature xyz". I'm mostly looking for input on missing features and use cases for those features. Including how the community would like to see the interfaces implemented for those.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">There's a lot of discussion about developers not always understanding use cases or use models for the end users so this seemed like an interesting opportunity to try and build something in cooperation with the Operators group. I don't know if this will work or not, but thought it would be worth a shot.</div>
<div style="font-family:'courier new',monospace"><br></div><div style="font-family:'courier new',monospace">Thanks,</div><div style="font-family:'courier new',monospace">
John</div><br></div></div>
<br>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br></blockquote></div><br></div></div></div>