Open Stack

On Tue, May 24, 2016 at 4:51 PM, Shrinand Javadekar <shrinand at maginatics.com
> wrote:

> Thanks for the detailed explanation...
>

Well, you're welcome - my apologies if it was overly verbose.


> This is unlike what I've seen in this setup. I have some code that
> tried to read the object 5 times from Swift with exponential backoff.
> But it failed with a 404 on all occassions which is why it gave up. I
> also tried manually using swift command line tool and got back an
> object "not found" error.
>

Well, that is really unfortunate - it sounds pretty broken - I can only
imagine that must be frustrating - but it unfortunately doesn't match any
of my experience or understanding.  I really think there's something
interesting about your configuration that I'm missing - but without the
transaction logs I really don't have anything concrete to investigate.


> The object was found in the *second* handoff node. Not the first. Does
> that matter.
>

No it wouldn't - again it's expected there would be *multiple* replicas -
possibly all on handoff nodes.  But even a single copy on a single handoff
would be enough to service the request, unless that node was not responding
for some other reason (mis-configuration of some kind?)


>
> The replicator eventually did transfer the blob to the original node.
> After that things were just fine...
>

Curious.  It shouldn't make a difference unless there was something wrong
with the handoff node?  Or perhaps more curious if for some reason the
proxy did not or could not contact that node in the read path?

Can you reliably reproduce this failure?  Is there any chance you can
capture logs from a request that 404's after a 201?  I don't think you
should have to "beat" replication - as long as you can get a 404 response
at some point; any point after that before logs rotate you should be able
to track down the txn_id and capture the needed logs.

I hope you're able to provide the needed logs.

Thanks,

-Clay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160524/f9701708/attachment.html>