<div dir="ltr">I managed to find out from SoftLayer that they're running 1.7.x</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Mar 13, 2014 at 11:39 AM, Ben McCann <span dir="ltr"><<a href="mailto:ben@benmccann.com" target="_blank">ben@benmccann.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The segments do appear to be there. <div><br></div><div>I don't see any X-Object-Manifest header though or anything like that</div>
<div><br></div><div><div> URL: <a href="https://dal05.objectstorage.softlayer.net/v1/AUTH_c4fd4142-xxxx-xxxx-xxxx-xxxxxxxxxxxx/mongodb-backups/mongodb_backup_2014.03.09.09.00.03.tar.gz" target="_blank">https://dal05.objectstorage.softlayer.net/v1/AUTH_c4fd4142-xxxx-xxxx-xxxx-xxxxxxxxxxxx/mongodb-backups/mongodb_backup_2014.03.09.09.00.03.tar.gz</a></div>
<div> Auth Token: AUTH_tkced5xxxxxxxxxxxxxxxxxxxxxxxxxxxx</div><div> Account: AUTH_c4fd4142-xxxx-xxxx-xxxx-xxxxxxxxxxxx</div><div> Container: mongodb-backups</div><div> Object: mongodb_backup_2014.03.09.09.00.03.tar.gz</div>
<div> Content Type: application/x-tar</div><div>Content Length: 3895</div><div> Last Modified: Sun, 09 Mar 2014 09:58:09 GMT</div><div> ETag: 0a8ec37fb235ac99e16a726e1b17375b</div><div> Meta Mtime: 1394358254.226578</div>
<div> Accept-Ranges: bytes</div><div> X-Timestamp: 1394359089.40223</div><div> X-Trans-Id: tx912c6e12d63a409aa5362b6075c5f930</div></div><div><br><div class="gmail_extra">Is this object likely to be stored wrong? Or maybe some misconfiguration on their end like a load banalancer stripping out headers or something?</div>
<div class="gmail_extra"><div><div class="h5"><br><br><div class="gmail_quote">On Thu, Mar 13, 2014 at 11:30 AM, Clay Gerrard <span dir="ltr"><<a href="mailto:clay.gerrard@gmail.com" target="_blank">clay.gerrard@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">No, you can't really turn off DLO and it's been around for awhile (let's assume for now they support it).<div>
<br></div><div>Can you verify the segments are showing up in the "container_segments" container? Can you verify the "X-Object-Manifest" header on the "container/file.tar.gz" object?</div>
<div><br></div><div>"swift stat -v container <span style="font-family:arial,sans-serif;font-size:12.800000190734863px">file.tar.gz" should show it as just "Manifest", something like "container</span>_segments/file.tar.gz/1394610073.993163/104857600/1000000/"</div>
<div><br></div><div>Good Luck,</div><div><br></div><div>-Clay</div>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Mar 13, 2014 at 11:00 AM, Ben McCann <span dir="ltr"><<a href="mailto:ben@benmccann.com" target="_blank">ben@benmccann.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Oh geeze, sorry, looking at those docs, it seems maybe I was mistaken and am using Dynamic Large Objects because I uploaded the file using swift -S. Is DLO also optional? Was it added at a certain version?<div>
<br></div><div>Thanks so much for the help!</div><div>-Ben</div></div><div class="gmail_extra"><div><div><br><br><div class="gmail_quote">On Thu, Mar 13, 2014 at 10:50 AM, Clay Gerrard <span dir="ltr"><<a href="mailto:clay.gerrard@gmail.com" target="_blank">clay.gerrard@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">You should check with your deployer to see what version of Swift they are running.<div>
<br></div><div>the /info (capabilities) feature was added in 1.11 and I think Havana shipped with 1.10</div>
<div><br></div>
<div>But I think SLO support has been around since 1.5 which should be in Havana, maybe even Grizzly. [1]</div><div><br></div><div>Either way SLO support has always been optional (and off by default) - so your deployer may only support DLO. [2]</div>
<div><br></div><div>1. <a href="https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#OpenStack_Object_Storage_.28Swift.29" target="_blank">https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#OpenStack_Object_Storage_.28Swift.29</a><br>
</div>
<div>2. <a href="http://docs.openstack.org/developer/swift/overview_large_objects.html" target="_blank">http://docs.openstack.org/developer/swift/overview_large_objects.html</a><br></div></div><div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">
On Thu, Mar 13, 2014 at 10:31 AM, Ben McCann <span dir="ltr"><<a href="mailto:ben@benmccann.com" target="_blank">ben@benmccann.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Oh, you're right, I did miss the -v option. Thanks!<div><br></div><div>The curl command isn't working for me, however.</div><div><br></div><div>StorageURL: <a href="https://dal05.objectstorage.softlayer.net/v1/AUTH_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" target="_blank">https://dal05.objectstorage.softlayer.net/v1/AUTH_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx</a><br>
</div><div><br></div><div><div>$ curl <a href="https://dal05.objectstorage.softlayer.net/info" target="_blank">https://dal05.objectstorage.softlayer.net/info</a></div><div>Bad URL</div></div><div><br></div><div><br></div>
<div><br></div></div><div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Mar 13, 2014 at 2:59 AM, Kuo Hugo <span dir="ltr"><<a href="mailto:tonytkdk@gmail.com" target="_blank">tonytkdk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Hi Ben, <div><br></div><div>1. What's the Swift CLI version are you using for now ? </div><div>2. The <font color="#ff0000">$>swift stat -v</font> suppose to show you the storage_url. -v means verbose. </div>
<div>3. There's another choice via curl </div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font color="#cccccc">~IRON_MAN$></font><font color="#ff9900">curl -i https://$FQDN_of_AUTH_SERVER/auth/v1.0 -H "X-auth-user:johndoe" -H "x-auth-key:password_of_john"</font></div>
<div><font color="#cccccc">HTTP/1.1 200 OK</font></div><div><font color="#cccccc">X-Storage-Url: </font><font color="#ff9900"><a href="https://swift.example.com" target="_blank">https://swift.example.com</a></font><font color="#cccccc">/v1/AUTH_johndoe</font></div>
<div><font color="#cccccc">X-Auth-Token: AUTH_tk0f4473fa1234567890</font></div><div><font color="#cccccc">Content-Type: text/html; charset=UTF-8</font></div><div><font color="#cccccc">Set-Cookie: X-Auth-Token=AUTH_tk0f4473fa1234567890; Path=/</font></div>
<div><font color="#cccccc">X-Storage-Token: AUTH_tk0f4473fa1234567890</font></div><div><font color="#cccccc">Content-Length: 0</font></div><div><font color="#cccccc">X-Trans-Id: txcfefd5396a0f4f4694707-0053217efd</font></div>
</blockquote><div><font color="#cccccc"><br></font></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font color="#000000">And then you can send a GET to </font><font color="#ff9900"><a href="https://swift.example.com/info" target="_blank">https://swift.example.com/info</a></font></div>
</blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font color="#999999">$> curl <a href="https://cloud.swiftstack.com/info" target="_blank">https://cloud.swiftstack.com/info</a></font></div>
<div><font color="#000000"><br>
</font></div><div><font color="#000000"><br></font></div></blockquote><font color="#000000">One more information here, the /info was available since Swift 1.11.0 </font><div><font color="#000000"><a href="https://github.com/openstack/swift/blob/master/CHANGELOG#L120-L125" target="_blank">https://github.com/openstack/swift/blob/master/CHANGELOG#L120-L125</a><br>
</font></div><div><font color="#000000"><br></font></div><div><font color="#000000"><br></font></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-03-13 14:41 GMT+08:00 Ben McCann <span dir="ltr"><<a href="mailto:ben@benmccann.com" target="_blank">ben@benmccann.com</a>></span>:<div>
<div><br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Thanks Samuel. That's super helpful. I didn't realize that SLO could be turned on/off on the server. I thought it was merely a client-side convention.<div>
<br></div><div>swift stat doesn't return a StorageUrl for me. It just returns:</div>
<div><div> Account:</div><div> Containers:</div><div> Objects:</div><div> Bytes:</div><div> Meta Cdn-Id:</div><div> Meta Nas-Id:</div><div> X-Timestamp:</div><div> X-Trans-Id:</div><div> Accept-Ranges:</div>
</div><div><br></div><div>I'll see if I can find out from someone at SoftLayer whether they have an old version or have large objects turned off unless you can think of any other ways I might be able to find out by querying the server.</div>
<div><br></div><div>Thanks,</div><div>Ben</div><div><br></div><div><br></div></div><div class="gmail_extra"><div><div><br><br><div class="gmail_quote">On Wed, Mar 12, 2014 at 11:09 PM, Samuel Merritt <span dir="ltr"><<a href="mailto:sam@swiftstack.com" target="_blank">sam@swiftstack.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div>On 3/12/14 9:14 PM, Ben McCann wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hi,<br>
<br>
I'm attempting to download a large file which I have uploaded to<br>
SoftLayer Object Storage using the swift tool. It's downloading only the<br>
manifest and is not downloading the segments. Any tips on figuring out<br>
what is going wrong?<br>
</blockquote>
<br></div>
It's entirely possible that the cluster in question lacks static large object support. Maybe they're running an old version of Swift (pre-1.8.0/Grizzly), or maybe they haven't enabled the static large object feature.<br>
<br>
What you'll need to do is probe the cluster to see if static large objects are enabled.<br>
<br>
The easy way is to politely ask the cluster with a GET request to /info. Run `swift stat -v`, then take the "StorageURL" field and change the path to be just "/info". Strip off any path that's already there.<br>
<br>
For example, if the output looks like<br>
<br>
StorageURL: <a href="http://192.168.22.2:8080/v1/AUTH_test" target="_blank">http://192.168.22.2:8080/v1/<u></u>AUTH_test</a><br>
Auth Token: AUTH_<u></u>tk89454677b67348e09eefc1d6fe4d<u></u>fa68<br>
Account: AUTH_test<br>
Containers: 1<br>
[...]<br>
<br>
then you'd simply run<br>
<br>
curl <a href="http://192.168.22.2:8080/info" target="_blank">http://192.168.22.2:8080/info</a><br>
<br>
The response will be a JSON hash, so maybe run it through a JSON pretty-printer, e.g.<br>
<br>
curl <a href="http://192.168.22.2:8080/info" target="_blank">http://192.168.22.2:8080/info</a> | python -mjson.tool<br>
<br>
On my development machine, the response looks something like this (some items removed for brevity):<br>
<br>
{<br>
"container_quotas": {},<br>
"formpost": {},<br>
"slo": { <---- LOOK FOR THIS<br>
"max_manifest_segments": 1000,<br>
"max_manifest_size": 2097152,<br>
"min_segment_size": 71<br>
},<br>
"swift": {<br>
"account_listing_limit": 10000,<br>
"container_listing_limit": 10000,<br>
"max_account_name_length": 256,<br>
"max_container_name_length": 256,<br>
"max_file_size": 5368709122,<br>
"max_meta_count": 90,<br>
"max_meta_name_length": 128,<br>
"max_meta_value_length": 256,<br>
"max_object_name_length": 1024,<br>
"version": "1.12.0.106.ga08e813"<br>
}<br>
}<br>
<br>
If you don't see SLO in there, then you're out of luck. Bug Softlayer and see if they'll turn it on.<br>
<br>
Now, it's also possible that you'll get a 412 response with the body "Bad URL" when you run the curl command above. That will tell you that the cluster is running 1.11.0 or earlier, but it won't tell you anything about SLO support. In that case, try uploading a normal file (cat picture or something) with the header "X-Static-Large-Object: yes". If that succeeds, then SLO is disabled; if SLO is enabled, then the PUT will fail.<br>
<br>
A more verbose description is available here: <a href="https://ask.openstack.org/en/question/7005/static-large-objects/" target="_blank">https://ask.openstack.org/en/<u></u>question/7005/static-large-<u></u>objects/</a><br>
<br>
<br>
______________________________<u></u>_________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack</a><br>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br><a href="http://about.me/benmccann" target="_blank">about.me/benmccann</a>
</font></span></div>
<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div></div></div><br></div>
<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><a href="http://about.me/benmccann" target="_blank">about.me/benmccann</a>
</div>
</div></div><br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br><a href="http://about.me/benmccann" target="_blank">about.me/benmccann</a>
</font></span></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br><a href="http://about.me/benmccann" target="_blank">about.me/benmccann</a>
</font></span></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><a href="http://about.me/benmccann" target="_blank">about.me/benmccann</a>
</div>