[Openstack] [Swift] block I/O all disks

Mark Kirkwood mark.kirkwood at catalyst.net.nz
Fri Nov 6 23:19:59 UTC 2015


Do you reboot the machine? It might be interesting just restart the 
swift storage services and see if that brings everything right again.

Also check out the swift logs for what it is doing when things start to 
hang, and also what dmesg is saying at the time.

A bit more info about your setup would be good - I'm guessing you have 
12 swift object devices (on sata) and ? account and container ones (on ssd)?

Approx how many containers and objects live on each server (or if easier 
tell us how many servers you have + replication level and how many 
accounts, containers and objects in total)?

Regards

Mark

On 04/11/15 22:41, Heiko Krämer wrote:
> Hi guys,
>
> we notice on our Swift storage nodes some problems with our disks.
> After some time they blocks all I/O requests to the disks. Therefore the
> server isn't working suddenly and needs a reboot.
>
> Serversetup:
> * Kernel 3.19.x
> * Ubuntu 14.04
> * Swift (Kilo)
> * 12 x 2TB SATA (JBOD)
> * 2 x 480GB SSD (Raid1)
> * 32GB RAM
> * 8 Cores CPU
>
> The first try was an upgrade of the raid controller firmware and
> drivers. The second one some tests of writes and reads to each disk.
> I can't reproduce this issue but i heard on the last summit, Swift can
> be the problem by this issue.
>
> Do anyone solve this problem ?
>
>
> - Heiko
>
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>





More information about the Openstack mailing list