Open Stack

Hello Clay,

 

Thanks for your answer it helped me to understand the challenge around this use case. Actually I thought it would not be straight forward because as you mentioned swift is driven by the partitions.

 

Anyhow we are about to provide an object store service powered by swift. The users see this service from their file perspective, so the questions I get ask are more likely like:

*         Does my file has all the necessary replications

*         If my file isn’t fully replicated yet what queue has it in the backlog

*         How long will it take until my file has all the necessary replicas

 

The user will accept that swift is not a real time replication and it’s based on the concept of eventual consistency, but does this exclude visibility about the internal processes and metrics.

 

Cheers

Chris

 

 

From: Clay Gerrard [mailto:clay.gerrard at gmail.com] 
Sent: Saturday, October 01, 2016 01:16
To: Chris <contact at progbau.de>
Cc: Openstack <openstack at lists.openstack.org>
Subject: Re: [Openstack] Swift replication backlog and replication status for specific file

 

To look at current availability of a single object you can use `swift-get-nodes` and check all of the primary locations - or if you have the `.data` file handy already you can use `swift-object-info`

 

Either of these options will tell you were the object should be, and also where it might be if there was a failure leading to handoff.  There's even some output to help you spot check those locations.  OTOH, if you have recently rebalanced you can use either of these tools with the old ring as well.

 

... but honestly unless your troubleshooting a specific failure these things may not work exactly according to your requirements.  They aren't really designed to be consumed on an "object-by-object" basis.  There's just too many objects!

 

As you pointed out swift tends to break things down by partition - and there's already a lot of partition-replicas!  (3 replicas * 2^16 partitions is a lot!)

 

One of the more recent ideas about how to visualize replication backlog is too look at all the partitions acctually on disk and see how it differs from the partitions that are assigned in the ring - some of the idea is fleshed out here:

 

http://docs.openstack.org/developer/swift/admin_guide.html#checking-handoff-partition-distribution

 

Maybe you have some different ideas?  Or a different use-case?  Or maybe it's a common goal that lots of deployments are solving in various ways or maybe something no one really currently has a good handle on? 

 

Are you working a solution to a particular problem in your deployment or anticipating a need down the road?

 

-Clay

 

On Fri, Sep 30, 2016 at 5:01 AM, Chris <contact at progbau.de <mailto:contact at progbau.de> > wrote:

Hello,

 

How is it possible to get the following information from swift:

-          Backlog of all replications based on files

-          Query the replication status for a specific file

 

I know there is the swift-dispersion-report but this gives the partition view. We are more interested in the file view

 

Thanks in advance.

 

Cheers,

Chris


_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org <mailto:openstack at lists.openstack.org> 
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20161003/aa7e656f/attachment.html>