<div dir="ltr">I think Sean and John are in the right direction. Nova and Cinder need to be more decoupled in the area of volume attachments.<div><br></div><div>I think some of the mess here is due to different Cinder backend behavior - with some Cinder backends you actually attach volumes to a host (e.g., FC, iSCSI), with some you attach to a VM (e.g., Ceph), and with some you attach an entire pool of volumes to a host (e.g., NFS). I think this difference should all be contained in the Nova drivers that do the attachments.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 11, 2016 at 6:06 AM, John Griffith <span dir="ltr"><<a href="mailto:john.griffith8@gmail.com" target="_blank">john.griffith8@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div class="h5"><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 10, 2016 at 5:12 PM, Fox, Kevin M <span dir="ltr"><<a href="mailto:Kevin.Fox@pnnl.gov" target="_blank">Kevin.Fox@pnnl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">But the issue is, when told to detach, some of the drivers do bad things. then, is it the driver's issue to refcount to fix the issue, or is it nova's to refcount so that it doesn't call the release before all users are done with it? I think solving it in the middle, in cinder's probably not the right place to track it, but if its to be solved on nova's side, nova needs to know when it needs to do it. But cinder might have to relay some extra info from the backend.<br>
<br>
Either way, On the driver side, there probably needs to be a mechanism on the driver to say it either can refcount properly so its multiattach compatible (or that nova should refcount), or to default to not allowing multiattach ever, so existing drivers don't break.<br>
<span><br>
Thanks,<br>
Kevin<br>
________________________________________<br>
From: Sean McGinnis [<a href="mailto:sean.mcginnis@gmx.com" target="_blank">sean.mcginnis@gmx.com</a>]<br>
</span>Sent: Wednesday, February 10, 2016 3:25 PM<br>
<span>To: OpenStack Development Mailing List (not for usage questions)<br>
Subject: Re: [openstack-dev] [Nova][Cinder] Multi-attach, determining when to call os-brick's connector.disconnect_volume<br>
<br>
</span><div><div>On Wed, Feb 10, 2016 at 11:16:28PM +0000, Fox, Kevin M wrote:<br>
> I think part of the issue is whether to count or not is cinder driver specific and only cinder knows if it should be done or not.<br>
><br>
> But if cinder told nova that particular multiattach endpoints must be refcounted, that might resolve the issue?<br>
><br>
> Thanks,<br>
> Kevin<br>
<br>
I this case (the point John and I were making at least) it doesn't<br>
matter. Nothing is driver specific, so it wouldn't matter which backend<br>
is being used.<br>
<br>
If a volume is needed, request it to be attached. When it is no longer<br>
needed, tell Cinder to take it away. Simple as that.<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div></div></div><div class="gmail_extra"><div class="gmail_default" style="font-family:monospace,monospace">Hey Kevin,</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">So I think what Sean M pointed out is still valid in your case. It's not really that some drivers do bad things, the problem is actually the way attach/detach works in OpenStack as a whole. The original design (which we haven't strayed very far from) was that you could only attach a single resource to a single compute node. That was it, there was no concept of multi-attach etc.</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">Now however folks want to introduce multi-attach, which means all of the old assumptions that the code was written on and designed around are kinda "bad assumptions" now. It's true, as you pointed out however that there are some drivers that behave or deal with targets in a way that makes things complicated, but they're completely inline with the scsi standards and aren't doing anything *wrong*.</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">The point Sean M and I were trying to make is that for the specific use case of a single volume being attached to a compute node, BUT being passed through to more than one Instance it might be worth looking at just ensuring that Compute Node doesn't call detach unless it's *done* with all of the Instances that it was passing that volume through to.</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">You're absolutely right, there are some *weird* things that a couple of vendors do with targets in the case of like replication where they may actually create a new target and attach; those sorts of things are ABSOLUTELY Cinder's problem and Nova should not have to know anything about that as a consumer of the Target.</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">My view is that maybe we should look at addressing the multiple use of a single target case in Nova, and then absolutely figure out how to make things work correctly on the Cinder side for all the different behaviors that may occur on the Cinder side from the various vendors.</div><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">Make sense?</div><span class="HOEnZb"><font color="#888888"><div class="gmail_default" style="font-family:monospace,monospace"><br></div><div class="gmail_default" style="font-family:monospace,monospace">John</div></font></span></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><b style="font-size:12.7272720336914px"><font color="#666666">Avishay Traeger, PhD</font></b><br></div><div><font color="#666666"><i>System Architect</i></font></div><div><span style="color:rgb(102,102,102);font-size:12.7272720336914px"><br></span></div><div><span style="color:rgb(102,102,102)">Mobile:</span><span style="color:rgb(102,102,102)"> </span><a value="+972524317955" style="color:rgb(17,85,204)">+972 54 447 1475</a><br></div><div><font color="#666666">E-mail: <a href="mailto:avishay@stratoscale.com" style="color:rgb(17,85,204)" target="_blank">avishay@stratoscale.com</a></font></div><div><font color="#666666"><br></font></div><div><img src="http://www.stratoscale.com/wp-content/uploads/Logo-Signature-Stratoscale-230.jpg"><br></div><div><font color="#666666"><br></font></div><div><p style="margin:0in"><a href="http://www.stratoscale.com/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:arial;font-size:9.75pt">Web</span></a><span style="font-family:arial;font-size:9.75pt"> | </span><a href="http://www.stratoscale.com/blog/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:arial;font-size:9.75pt">Blog</span></a><span style="font-family:arial;font-size:9.75pt;color:rgb(108,163,214)"> | </span><a href="https://twitter.com/Stratoscale" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:arial;font-size:9.75pt">Twitter</span></a><span style="font-family:arial;font-size:9.75pt;color:rgb(108,163,214)"> | <a href="https://plus.google.com/u/1/b/108421603458396133912/108421603458396133912/posts" style="color:rgb(17,85,204)" target="_blank">Google+</a> | </span><span style="font-family:arial;font-size:9.75pt"><a href="https://www.linkedin.com/company/stratoscale" style="color:rgb(17,85,204)" target="_blank">Linkedin</a></span></p></div></div></div></div></div></div></div></div></div>
</div>