[openstack-dev] [swift] Swift container sync process is consuming 100% CPU

Alan Jiang ajiang at linux.vnet.ibm.com
Wed Jul 15 17:43:11 UTC 2015

In our production swift cluster, we have container sync middleware enabled in the pipeline. But
swift container sync is not configured. 

We noticed 100% CPU utilization and a lot of read/write i/os from the swift-container-sync process
on our meta data nodes recently after we have seen increasing volumes of POST/DELETE requests.

From the code it makes sense since container_sync() calls broker.get_info() which will merge the
pending requests to the container db before it gets the metadata from the container_stat table. 

I have two questions:

1. Is it safe just to stop the swift-container-sync process on all of our metadata nodes(have all the account/container daemons running)?
Any side effect if we do so?

2. Since container_updater is doing the same broker.get_info() in the container_sweep() code path, and the concurrency is 4, why
don’t I see swift-container-updater sitting on the top CPU consumer list?  My assumption is due to each swift-container-updater child
process is forked to handle only one partition per device. So it is a short lived process which won’t be captured by atop or top. 
Is that true?

Alan Jiang

More information about the OpenStack-dev mailing list