<div dir="ltr">There may have someone can provide a explanation in detail. <div>Seems there're various cases. I just checked the code of Proxy's object controller. </div><div>In case of connection failed to a target object device, seems the request will be stop at <a href="https://github.com/openstack/swift/blob/master/swift/proxy/controllers/obj.py#L1091">https://github.com/openstack/swift/blob/master/swift/proxy/controllers/obj.py#L1091</a> .</div>
<div><br></div><div>I'm awaiting for more accurate answers. </div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><div><br></div><div>+Hugo Kuo+</div><div>(+886) 935004793<br>
</div><div>SwiftStack Inc.<br></div></div></div>
<br><br><div class="gmail_quote">2013/11/29 Shao, Minglong <span dir="ltr"><<a href="mailto:Minglong.Shao@netapp.com" target="_blank">Minglong.Shao@netapp.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>Thanks Hugo for your reply!</div>
<div><br>
</div>
<div>It’s still very puzzling. Each object server handles the request independently. </div>
<div>How can the proxy interrupt the PUT request to the good device? </div>
<div>The line of code you referred to does checking before the object server actually writes anything.</div>
<div>The exception is raised only if the device is not available.</div>
<div>By the time the proxy knows that the object PUT is a failure, the good object server has most likely passed that checking.</div>
<div>Besides, I don’t see any proxy code that sends “interrupt” to object servers after it checks quorum.</div>
<div><br>
</div>
<div>Regards,</div>
<div>-Minglong</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Kuo Hugo <<a href="mailto:tonytkdk@gmail.com" target="_blank">tonytkdk@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, November 28, 2013 at 5:46 PM<div><div class="h5"><br>
<span style="font-weight:bold">To: </span>Minglong Shao <<a href="mailto:minglong.shao@netapp.com" target="_blank">minglong.shao@netapp.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>" <<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Openstack] [Swift] Container DB update after object PUT<br>
</div></div></div><div><div class="h5">
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div>Proxy will interrupt the PUT request to the only good device. </div>
<div><br>
</div>
<div>Object server returned error to Proxy</div>
<a href="https://github.com/openstack/swift/blob/master/swift/obj/server.py#L387" target="_blank">https://github.com/openstack/swift/blob/master/swift/obj/server.py#L387</a><br>
<div><br>
</div>
<div><br>
</div>
<div>
<div><font size="1" color="#999999">account-server 192.168.56.10 - - "HEAD /d5/390/AUTH_ss" 204 - "txdc2f2cc49b7b447e91f93-0052970cca" "HEAD
<a href="http://192.168.56.10/v1/AUTH_ss" target="_blank">http://192.168.56.10/v1/AUTH_ss</a>" "proxy-server 23471" 0.0012 ""</font></div>
<div><font size="1" color="#999999">container-server 192.168.56.10 - - "HEAD /d3/408/AUTH_ss/con1" 204 - "txdc2f2cc49b7b447e91f93-0052970cca" "HEAD
<a href="http://192.168.56.10/v1/AUTH_ss/con1" target="_blank">http://192.168.56.10/v1/AUTH_ss/con1</a>" "proxy-server 23471" 0.0011</font></div>
<div><span style="font-size:x-small"><font color="#999999">object-server 192.168.56.10 - - "PUT /d2/293/AUTH_ss/con1/8" 507 - "PUT
<a href="http://192.168.56.10/v1/AUTH_ss/con1/8" target="_blank">http://192.168.56.10/v1/AUTH_ss/con1/8</a>" "txdc2f2cc49b7b447e91f93-0052970cca" "proxy-server 23471" 0.0004</font></span></div>
<div><font size="1" color="#999999">proxy-server ERROR Insufficient Storage <a href="http://192.168.56.10:6000/d2" target="_blank">
192.168.56.10:6000/d2</a> (txn: txdc2f2cc49b7b447e91f93-0052970cca)</font></div>
<div><font size="1" color="#999999">object-server 192.168.56.10 - - "PUT /d1/293/AUTH_ss/con1/8" 507 - "PUT
<a href="http://192.168.56.10/v1/AUTH_ss/con1/8" target="_blank">http://192.168.56.10/v1/AUTH_ss/con1/8</a>" "txdc2f2cc49b7b447e91f93-0052970cca" "proxy-server 23471" 0.0002</font></div>
<div><font size="1" color="#999999">proxy-server ERROR Insufficient Storage <a href="http://192.168.56.10:6000/d1" target="_blank">
192.168.56.10:6000/d1</a> (txn: txdc2f2cc49b7b447e91f93-0052970cca)</font></div>
<div><font size="1" color="#999999">proxy-server Object PUT returning 503, 1/2 required connections (txn: txdc2f2cc49b7b447e91f93-0052970cca)</font></div>
<div><font size="1" color="#999999">object-server 192.168.56.10 - - "PUT /d0/293/AUTH_ss/con1/8"
</font><font size="1" color="#ff0000">499</font><font size="1" color="#999999"> - "PUT
<a href="http://192.168.56.10/v1/AUTH_ss/con1/8" target="_blank">http://192.168.56.10/v1/AUTH_ss/con1/8</a>" "txdc2f2cc49b7b447e91f93-0052970cca" "proxy-server 23471" 0.0050</font></div>
<div><font size="1" color="#999999">proxy-server 192.168.56.10 192.168.56.10 28/Nov/2013/09/28/42 PUT /v1/AUTH_ss/con1/8 HTTP/1.0 503 - curl/7.22.0%20%28x86_64-pc-linux-gnu%29%20libcurl/7.22.0%20OpenSSL/1.0.1%20zlib/<a href="http://1.2.3.4" target="_blank">1.2.3.4</a>%20libidn/1.23%20librtmp/2.3
ss%2CAUTH_tk8caa909045a940aca3ff980ffa507961 - 118 - txdc2f2cc49b7b447e91f93-0052970cca - 0.0145 - -</font></div>
</div>
<div><font size="1" color="#999999"><br>
</font></div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div dir="ltr">
<div><br>
</div>
<div>+Hugo Kuo+</div>
<div><a href="tel:%28%2B886%29%20935004793" value="+886935004793" target="_blank">(+886) 935004793</a><br>
</div>
<div>SwiftStack Inc.<br>
</div>
</div>
</div>
<br>
<br>
<div class="gmail_quote">2013/11/28 Shao, Minglong <span dir="ltr"><<a href="mailto:Minglong.Shao@netapp.com" target="_blank">Minglong.Shao@netapp.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>Thanks for your reply!</div>
<div>I understand the three PUTs by the proxy server and how the replicator works.</div>
<div><br>
</div>
<div>What I don’t understand is the update of the container DB. The update is sent by individual object servers which don’t know whether a PUT (from the client’s perspective) succeeds. </div>
<div>Consider the following scenario:</div>
<ol>
<li>Proxy server sends three requests to three object servers.</li><li>One object server writes the object successfully, sends an update to the container DB and an “OK” reply to the proxy server. But the other two fail, so they send “failed” to the proxy server.
</li><li>The proxy server sends back “failed” to the client because it doesn’t meet the quorum. But the container DB still gets the update to insert an entry of this object.</li></ol>
<div>I must have missed something. Thanks for your help!</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Kuo Hugo <<a href="mailto:tonytkdk@gmail.com" target="_blank">tonytkdk@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, November 28, 2013 at 12:31 PM<br>
<span style="font-weight:bold">To: </span>Minglong Shao <<a href="mailto:minglong.shao@netapp.com" target="_blank">minglong.shao@netapp.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>" <<a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Openstack] [Swift] Container DB update after object PUT<br>
</div>
<div>
<div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">Proxy sends requests to *three* replicas's object servers simultaneously. <br>
A successful PUT request depends on at least 1/2 replicas were success. Or Proxy will return PUT failed to user.
<div><br>
</div>
<div>If 1/3 replica is not there, the replicator will handle it later. </div>
<div>It won't have any problem with container DB. </div>
<div><br>
</div>
<div> </div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div dir="ltr">
<div><br>
</div>
<div>+Hugo Kuo+</div>
<div><a href="tel:%28%2B886%29%20935004793" value="+886935004793" target="_blank">(+886) 935004793</a><br>
</div>
<div>SwiftStack Inc.<br>
</div>
</div>
</div>
<br>
<br>
<div class="gmail_quote">2013/11/28 Shao, Minglong <span dir="ltr"><<a href="mailto:Minglong.Shao@netapp.com" target="_blank">Minglong.Shao@netapp.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>
<div>Hi there,</div>
<div><br>
</div>
<div>After an object server writes an object in the local file system, it updates the container DB asynchronously (send a message to insert an entry in the object table). But the object server doesn’t really know whether the object PUT is considered successful
or not because the other two replicas could fail. In this case, the container DB could have an entry for an object which is not successfully PUT. Can someone shed some light on this? Am I missing something?</div>
<div><br>
</div>
<div>Many thanks!</div>
</div>
</div>
<br>
_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</span></div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div></div></span>
</div>
</blockquote></div><br></div>