Open Stack

Hello, All

I have a terrible problem with object server.
Here is the case.
1. User upload an object to proxy-server
2. Proxy server try to connect with object-server
3. If one of object-server is slow to respond, proxy-server is waiting for
response.
  3.1 While waiting for response, proxy-server can't do anything
4. So, The response of client request will be delayed.

In my opinion, this code seems to be a problem
(
https://github.com/openstack/swift/blob/stable/rocky/swift/proxy/controllers/obj.py#L1734
)

```
with ResponseTimeout(node_timeout):
            resp = conn.getexpect()
```

If node_timeout's value is 3 and object-server respond after 2 seconds,
proxy-server wait 2 seconds.

Because proxy-server wait for the above response, the execution of the
following code is delayed.
(
https://github.com/openstack/swift/blob/stable/rocky/swift/proxy/controllers/obj.py#L627
)

```
for node in nodes:
            try:
                putter = self._make_putter(node, part, req, headers)
                self.app.set_node_timing(node, putter.connect_duration)
                return putter
```

This problem occurs when i do a ring rebalance.
When object-replicator delete a partition directory that are no longer
mine, the disk becomes very busy (Because of xfsaild daemon)
Because the disk are busy, object-server can't create diskfile during PUT
operation.

Is there anyone who is having problems like me?
How can I solve this problem?

I need everyone's help.
Thanks.

Best Regards
SeongSoo Cho

------
SeongSoo Cho (South Korea)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20181029/dc319bb1/attachment.html>