<div dir="ltr">Thanks Matthew for your reply<div><br></div><div>Problem is resolved now both object and container ring was using same port. Changing the port in ring configuration fixed the issue and we are able to upload the files now.</div><div><br></div><div>Aravind M D</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 13, 2018 at 3:59 AM, Matthew Oliver <span dir="ltr"><<a href="mailto:matt@oliver.net.au" target="_blank">matt@oliver.net.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi Aravind,</div><div><br></div><div>There only seems to be container-server logs in your reply. So you have any from the object server?<br></div><div>Also what ports are your object-server and container-servers listening on? Your ring is saying the object-servers are listening on 6201. Just making sure the port numbers aren't confused and the proxy isn't tryng to send the object PUTs to the container servers. Because it's interesting that there is no objects subfolder.</div><div><br></div><div>Matt</div><div><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 12, 2018 at 10:12 PM, aRaviNd <span dir="ltr"><<a href="mailto:ambadiaravind@gmail.com" target="_blank">ambadiaravind@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Clay, Hi All,<div><br></div><div>Configured my cluster from ground up with one proxy node and one storage node.</div><div><br></div><div><img src="cid:ii_jdk432nb0_16189b01a4e8209a" width="562" height="102"><br>​<br></div><div>Now I am getting two types of errors.</div><div><br></div><div><div>Large files:</div><div><br></div><div>Feb  3 18:12:33 centos7-swift-proxy1 swift-proxy-server: ERROR with Object server <a href="http://192.168.47.128:6201/sda" target="_blank">192.168.47.128:6201/sda</a> re: Trying to write to /AUTH_admin/ara1/abc: #012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/controllers/<wbr>obj.py", line 1617, in _send_file#012    self.conn.send(to_send)#012  File "/usr/lib64/python2.7/httplib.<wbr>py", line 840, in send#012    self.sock.sendall(data)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/eventlet/greenio/base.py", line 393, in sendall#012    tail += self.send(data[tail:], flags)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/eventlet/greenio/base.py", line 384, in send#012    return self._send_loop(self.fd.send, data, flags)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/eventlet/greenio/base.py", line 371, in _send_loop#012    return send_method(data, *args)#012error: [Errno 32] Broken pipe</div><div>Feb  3 18:12:33 centos7-swift-proxy1 swift-proxy-server: Object 1 PUT exceptions during send, 0/1 required connections (txn: tx0b01a226078f47b4b3593-005a76<wbr>41e1) (client_ip: 192.168.47.132)</div></div><div><br></div><div>Large file error on storage node:</div><div><br></div><div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35624)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:15:51 +0000] "PUT /sda/89/AUTH_admin/ara1" 202 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1</a>" "tx158875c238214209bdfc3-005a7<wbr>646ab" "proxy-server 44255" 0.0063 "-" 112666 0</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:15:51] "PUT /sda/89/AUTH_admin/ara1 HTTP/1.1" 202 252 0.006706 (txn: tx158875c238214209bdfc3-005a76<wbr>46ab)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35630)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:15:51 +0000] "HEAD /sda/89/AUTH_admin/ara1" 204 - "HEAD <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1</a>" "txc23a59c70e8940bf85101-005a7<wbr>646ab" "proxy-server 44253" 0.0011 "-" 112666 0</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:15:51] "HEAD /sda/89/AUTH_admin/ara1 HTTP/1.1" 204 521 0.001422 (txn: txc23a59c70e8940bf85101-005a76<wbr>46ab)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35632)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:15:51 +0000] "HEAD /sda/75/AUTH_admin/ara1/abc" 404 - "HEAD <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/abc" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/abc</a>" "txc23a59c70e8940bf85101-005a7<wbr>646ab" "proxy-server 44253" 0.0003 "-" 112666 0</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:15:51] "HEAD /sda/75/AUTH_admin/ara1/abc HTTP/1.1" 404 351 0.000549 (txn: txc23a59c70e8940bf85101-005a76<wbr>46ab)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35634)</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:15:51 +0000] "PUT /sda/75/AUTH_admin/ara1/abc" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/abc" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/abc</a>" "tx5138df5cf1f840808d2b2-005a7<wbr>646ab" "proxy-server 44253" 0.0002 "-" 112666 0</div><div>Feb  4 19:15:51 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:15:51] "PUT /sda/75/AUTH_admin/ara1/abc HTTP/1.1" 404 212 0.000506 (txn: tx5138df5cf1f840808d2b2-005a76<wbr>46ab)</div><div>Feb  4 19:15:52 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35636)</div><div>Feb  4 19:15:52 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:15:52 +0000] "PUT /sda/75/AUTH_admin/ara1/abc" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/abc" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/abc</a>" "txeb1c76b896854e4885009-005a7<wbr>646ac" "proxy-server 44253" 0.0002 "-" 112666 0</div><div>Feb  4 19:15:52 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:15:52] "PUT /sda/75/AUTH_admin/ara1/abc HTTP/1.1" 404 212 0.000589 (txn: txeb1c76b896854e4885009-005a76<wbr>46ac)</div><div>Feb  4 19:15:54 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35638)</div><div>Feb  4 19:15:54 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:15:54 +0000] "PUT /sda/75/AUTH_admin/ara1/abc" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/abc" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/abc</a>" "tx1a011b125608447ca36a8-005a7<wbr>646ae" "proxy-server 44253" 0.0002 "-" 112666 0</div><div>Feb  4 19:15:54 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:15:54] "PUT /sda/75/AUTH_admin/ara1/abc HTTP/1.1" 404 212 0.000569 (txn: tx1a011b125608447ca36a8-005a76<wbr>46ae)</div></div><div><br></div><div>or</div><div><br></div><div>Smaller files:</div><div><br></div><div>Feb  3 18:30:40 centos7-swift-proxy1 swift-proxy-server: ERROR Unhandled exception in request: #012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/server.py", line 521, in handle_request#012    return handler(req)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/controllers/<wbr>base.py", line 283, in wrapped#012    return func(*a, **kw)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/controllers/<wbr>obj.py", line 745, in PUT#012    req, data_source, nodes, partition, outgoing_headers)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/controllers/<wbr>obj.py", line 949, in _store_object#012    self._get_put_responses(req, putters, len(nodes))#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/controllers/<wbr>obj.py", line 406, in _get_put_responses#012    _handle_response(putter, response)#012  File "/usr/lib/python2.7/site-packa<wbr>ges/swift/proxy/controllers/<wbr>obj.py", line 402, in _handle_response#012    etags.add(response.getheader('<wbr>etag').strip('"'))#012Attribut<wbr>eError: 'NoneType' object has no attribute 'strip' (txn: tx029f271d16a84de09d4fb-005a76<wbr>4620) (client_ip: 192.168.47.132)<br></div><div><br></div><div>Smaller file error on storage node:</div><div><br></div><div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35696)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:34 +0000] "PUT /sda/89/AUTH_admin/ara1" 202 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1</a>" "txb8d792ec40844162978cc-005a7<wbr>64712" "proxy-server 44253" 0.0077 "-" 112666 0</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:34] "PUT /sda/89/AUTH_admin/ara1 HTTP/1.1" 202 252 0.008124 (txn: txb8d792ec40844162978cc-005a76<wbr>4712)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35702)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:34 +0000] "HEAD /sda/89/AUTH_admin/ara1" 204 - "HEAD <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1</a>" "tx672029e079574e78b1c1b-005a7<wbr>64712" "proxy-server 44254" 0.0011 "-" 112666 0</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:34] "HEAD /sda/89/AUTH_admin/ara1 HTTP/1.1" 204 521 0.001465 (txn: tx672029e079574e78b1c1b-005a76<wbr>4712)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35704)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:34 +0000] "HEAD /sda/26/AUTH_admin/ara1/swift.<wbr>conf" 404 - "HEAD <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/swift.conf" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/swift.conf</a>" "tx672029e079574e78b1c1b-005a7<wbr>64712" "proxy-server 44254" 0.0003 "-" 112666 0</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:34] "HEAD /sda/26/AUTH_admin/ara1/swift.<wbr>conf HTTP/1.1" 404 351 0.000547 (txn: tx672029e079574e78b1c1b-005a76<wbr>4712)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35706)</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:34 +0000] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/swift.conf" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/swift.conf</a>" "tx0ec94217b5584b7ea4e21-005a7<wbr>64712" "proxy-server 44254" 0.0002 "-" 112666 0</div><div>Feb  4 19:17:34 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:34] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf HTTP/1.1" 404 212 0.000513 (txn: tx0ec94217b5584b7ea4e21-005a76<wbr>4712)</div><div>Feb  4 19:17:35 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35708)</div><div>Feb  4 19:17:35 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:35 +0000] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/swift.conf" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/swift.conf</a>" "tx8e93f98453804972b39d8-005a7<wbr>64713" "proxy-server 44254" 0.0004 "-" 112666 0</div><div>Feb  4 19:17:35 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:35] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf HTTP/1.1" 404 212 0.000791 (txn: tx8e93f98453804972b39d8-005a76<wbr>4713)</div><div>Feb  4 19:17:37 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35710)</div><div>Feb  4 19:17:37 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:37 +0000] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/swift.conf" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/swift.conf</a>" "txd44faaca5b5b408da4fea-005a7<wbr>64715" "proxy-server 44254" 0.0004 "-" 112666 0</div><div>Feb  4 19:17:37 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:37] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf HTTP/1.1" 404 212 0.000716 (txn: txd44faaca5b5b408da4fea-005a76<wbr>4715)</div><div>Feb  4 19:17:41 centos7-swift-node1 container-server: STDERR: (112666) accepted ('192.168.47.132', 35712)</div><div>Feb  4 19:17:41 centos7-swift-node1 container-server: 192.168.47.132 - - [05/Feb/2018:00:17:41 +0000] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf" 404 - "PUT <a href="http://192.168.47.132:8080/v1/AUTH_admin/ara1/swift.conf" target="_blank">http://192.168.47.132:8080/v1/<wbr>AUTH_admin/ara1/swift.conf</a>" "tx6e5c49ba6d54489aa1de5-005a7<wbr>64719" "proxy-server 44254" 0.0002 "-" 112666 0</div><div>Feb  4 19:17:41 centos7-swift-node1 container-server: STDERR: 192.168.47.132 - - [05/Feb/2018 00:17:41] "PUT /sda/26/AUTH_admin/ara1/swift.<wbr>conf HTTP/1.1" 404 212 0.000590 (txn: tx6e5c49ba6d54489aa1de5-005a76<wbr>4719)</div></div><div><br></div><div>Directories created on storage node are</div><div><br></div><div><img src="cid:ii_jdk4ca2o1_16189b6a97f43f37" width="360" height="153"><br><br></div><div>We are really stuck on the issue and not able to use swift in our environment. Any help will be really appreciated.​<br></div><div><br></div><div>Aravind M D</div></div><div class="m_-3840465740343524919HOEnZb"><div class="m_-3840465740343524919h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 8, 2018 at 2:43 AM, Clay Gerrard <span dir="ltr"><<a href="mailto:clay.gerrard@gmail.com" target="_blank">clay.gerrard@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">One replica is a little strange.  Do the uploads *always* fail - in the same way?  Or is this just one example of a PUT that returned 503?  Are you doing a lot of concurrent PUTs to the same object/name/disk?</span><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">The error from the log (EPIPE) means the object-server closed the connection as the proxy was writing to it... which is a little strange.  There should be a corresponding exception/error from the object-server service - you should make sure the object-servers are running and find where they are logging - then grep all the logs for the transaction-id to get a better picture of the whole distributed transaction.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">If you keep digging I know you can find the problem.  Let us know what you find.</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">Good luck,</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">-Clay</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_-3840465740343524919m_9157195006759585006h5">On Wed, Feb 7, 2018 at 1:58 AM, aRaviNd <span dir="ltr"><<a href="mailto:ambadiaravind@gmail.com" target="_blank">ambadiaravind@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-3840465740343524919m_9157195006759585006h5"><div dir="ltr"><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Hi All,</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">We have created an openstack cluster with one proxy server and three storage nodes. Configuration consist of two regions and three zones.</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><a href="https://i.stack.imgur.com/w9EWG.png" rel="nofollow noreferrer" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(0,89,153);text-decoration:none" target="_blank"><img src="https://i.stack.imgur.com/w9EWG.png" alt="enter image description here" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;max-width:100%"></a></p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">We are able to create containers</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><a href="https://i.stack.imgur.com/QgQQM.png" rel="nofollow noreferrer" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(0,89,153);text-decoration:none" target="_blank"><img src="https://i.stack.imgur.com/QgQQM.png" alt="enter image description here" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;max-width:100%"></a></p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">But while trying to upload files we are getting 503 service unavailable and seeing below logs in swift.log</p><p style="margin:0px 0px 1em;padding:0px;border:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:15px;line-height:inherit;font-family:Arial,"Helvetica Neue",Helvetica,sans-serif;vertical-align:baseline;clear:both;color:rgb(36,39,41);letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><a href="https://i.stack.imgur.com/7ROMm.png" rel="nofollow noreferrer" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;color:rgb(0,89,153);text-decoration:none" target="_blank"><img src="https://i.stack.imgur.com/7ROMm.png" alt="enter image description here" style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inherit;font-stretch:inherit;font-size:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;max-width:100%"></a></p><span class="m_-3840465740343524919m_9157195006759585006m_-6374415619775090641HOEnZb"><font color="#888888">- Aravind </font></span></div>
<br></div></div>______________________________<wbr>_________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k</a><br>
Post to     : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k</a><br>
Post to     : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k</a><br>
<br></blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>