<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Chris,<br>
</p>
<br>
<div class="moz-cite-prefix">On 10/03/2016 01:21 AM, Chris wrote:<br>
</div>
<blockquote cite="mid:005201d21d36$0706eea0$1514cbe0$@progbau.de"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.m3566890852407396156msolistparagraph, li.m3566890852407396156msolistparagraph, div.m3566890852407396156msolistparagraph
{mso-style-name:m_3566890852407396156msolistparagraph;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:789326513;
mso-list-type:hybrid;
mso-list-template-ids:-489934068 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hello
Clay,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thanks
for your answer it helped me to understand the challenge
around this use case. Actually I thought it would not be
straight forward because as you mentioned swift is driven by
the partitions.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Anyhow
we are about to provide an object store service powered by
swift. The users see this service from their file
perspective, so the questions I get ask are more likely
like:<o:p></o:p></span></p>
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span
style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span
style="mso-list:Ignore">·<span style="font:7.0pt
"Times New Roman""> </span></span></span><!--[endif]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Does
my file has all the necessary replications<o:p></o:p></span></p>
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span
style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span
style="mso-list:Ignore">·<span style="font:7.0pt
"Times New Roman""> </span></span></span><!--[endif]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">If
my file isn’t fully replicated yet what queue has it in the
backlog<o:p></o:p></span></p>
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span
style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span
style="mso-list:Ignore">·<span style="font:7.0pt
"Times New Roman""> </span></span></span><!--[endif]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">How
long will it take until my file has all the necessary
replicas</span></p>
</div>
</blockquote>
<blockquote cite="mid:005201d21d36$0706eea0$1514cbe0$@progbau.de"
type="cite">
<div class="WordSection1">
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">The
user will accept that swift is not a real time replication
and it’s based on the concept of eventual consistency, but
does this exclude visibility about the internal processes
and metrics.</span></p>
</div>
</blockquote>
I *think* what you are getting at is a misconception about how Swift
and eventual consistency works. I've heard that before where people
think in the case of swift, eventual consistency means that when a
user sends a PUT request to swift, *one* copy is written to disk and
that there's a background process replicating the object to make it
durable. This is not the case...<br>
<br>
Swift will only return a success message to the user, once the
object has been durably written to disk. Durably means at least a
quorum has been reached. In the case of 3x replication, a user will
receive a success message (HTTP 201 Created) when at least 2 copies
of the object has been written (and fsynced) to disk. <br>
<br>
When the swift proxy is unable to write data to a primary device, it
will write the data to a hand-off device. In this case, the
background replication process will attempt to move that object to
the primary device once the problem is fixed. During failure
scenarios like this, it is possible to read stale data, but Swift
will eventually move data to primary nodes so that the correct, most
current data is always read.<br>
<br>
I hope this helps, the docs [1] also has a ton of useful info...<br>
<br>
Regards,<br>
<br>
Thiago<br>
<br>
[1] -
<a class="moz-txt-link-freetext" href="http://docs.openstack.org/developer/swift/overview_architecture.html">http://docs.openstack.org/developer/swift/overview_architecture.html</a><br>
<br>
<br>
<blockquote cite="mid:005201d21d36$0706eea0$1514cbe0$@progbau.de"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Cheers<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Chris<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif">
Clay Gerrard [<a class="moz-txt-link-freetext" href="mailto:clay.gerrard@gmail.com">mailto:clay.gerrard@gmail.com</a>] <br>
<b>Sent:</b> Saturday, October 01, 2016 01:16<br>
<b>To:</b> Chris <a class="moz-txt-link-rfc2396E" href="mailto:contact@progbau.de"><contact@progbau.de></a><br>
<b>Cc:</b> Openstack <a class="moz-txt-link-rfc2396E" href="mailto:openstack@lists.openstack.org"><openstack@lists.openstack.org></a><br>
<b>Subject:</b> Re: [Openstack] Swift replication backlog
and replication status for specific file<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">To look at current availability of a
single object you can use `swift-get-nodes` and check all of
the primary locations - or if you have the `.data` file
handy already you can use `swift-object-info`<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Either of these options will tell you
were the object should be, and also where it might be if
there was a failure leading to handoff. There's even some
output to help you spot check those locations. OTOH, if
you have recently rebalanced you can use either of these
tools with the old ring as well.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">... but honestly unless your
troubleshooting a specific failure these things may not
work exactly according to your requirements. They aren't
really designed to be consumed on an "object-by-object"
basis. There's just too many objects!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">As you pointed out swift tends to break
things down by partition - and there's already a lot of
partition-replicas! (3 replicas * 2^16 partitions is a
lot!)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">One of the more recent ideas about how
to visualize replication backlog is too look at all the
partitions acctually on disk and see how it differs from
the partitions that are assigned in the ring - some of the
idea is fleshed out here:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><a moz-do-not-send="true"
href="http://docs.openstack.org/developer/swift/admin_guide.html#checking-handoff-partition-distribution">http://docs.openstack.org/developer/swift/admin_guide.html#checking-handoff-partition-distribution</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Maybe you have some different ideas?
Or a different use-case? Or maybe it's a common goal that
lots of deployments are solving in various ways or maybe
something no one really currently has a good handle on? <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Are you working a solution to a
particular problem in your deployment or anticipating a
need down the road?<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-Clay<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Fri, Sep 30, 2016 at 5:01 AM, Chris
<<a moz-do-not-send="true"
href="mailto:contact@progbau.de" target="_blank">contact@progbau.de</a>>
wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hello,<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">How
is it possible to get the following information from
swift:<o:p></o:p></p>
<p class="m3566890852407396156msolistparagraph">-<span
style="font-size:7.0pt"> </span>Backlog
of all replications based on files<o:p></o:p></p>
<p class="m3566890852407396156msolistparagraph">-<span
style="font-size:7.0pt"> </span>Query
the replication status for a specific file<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I
know there is the swift-dispersion-report but this
gives the partition view. We are more interested in
the file view<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks
in advance.<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Cheers,<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Chris<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Mailing list: <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a moz-do-not-send="true"
href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Mailing list: <a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
Post to : <a class="moz-txt-link-abbreviated" href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a>
Unsubscribe : <a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
</pre>
</blockquote>
<br>
</body>
</html>