[Openstack] Swift downloading only manifest of static large object (SLO)

Ben McCann ben at benmccann.com
Thu Mar 13 18:00:57 UTC 2014


Oh geeze, sorry, looking at those docs, it seems maybe I was mistaken and
am using Dynamic Large Objects because I uploaded the file using swift -S.
Is DLO also optional? Was it added at a certain version?

Thanks so much for the help!
-Ben


On Thu, Mar 13, 2014 at 10:50 AM, Clay Gerrard <clay.gerrard at gmail.com>wrote:

> You should check with your deployer to see what version of Swift they are
> running.
>
> the /info (capabilities) feature was added in 1.11 and I think Havana
> shipped with 1.10
>
> But I think SLO support has been around since 1.5 which should be in
> Havana, maybe even Grizzly. [1]
>
> Either way SLO support has always been optional (and off by default) - so
> your deployer may only support DLO. [2]
>
> 1.
> https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#OpenStack_Object_Storage_.28Swift.29
> 2. http://docs.openstack.org/developer/swift/overview_large_objects.html
>
>
> On Thu, Mar 13, 2014 at 10:31 AM, Ben McCann <ben at benmccann.com> wrote:
>
>> Oh, you're right, I did miss the -v option. Thanks!
>>
>> The curl command isn't working for me, however.
>>
>> StorageURL:
>> https://dal05.objectstorage.softlayer.net/v1/AUTH_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
>>
>> $ curl https://dal05.objectstorage.softlayer.net/info
>> Bad URL
>>
>>
>>
>>
>>
>> On Thu, Mar 13, 2014 at 2:59 AM, Kuo Hugo <tonytkdk at gmail.com> wrote:
>>
>>> Hi Ben,
>>>
>>> 1. What's the Swift CLI version are you using for now ?
>>> 2. The $>swift stat -v suppose to show you the storage_url. -v means
>>> verbose.
>>> 3. There's another choice via curl
>>>
>>> ~IRON_MAN$>curl -i https://$FQDN_of_AUTH_SERVER/auth/v1.0 -H
>>> "X-auth-user:johndoe" -H "x-auth-key:password_of_john"
>>> HTTP/1.1 200 OK
>>> X-Storage-Url: https://swift.example.com/v1/AUTH_johndoe
>>> X-Auth-Token: AUTH_tk0f4473fa1234567890
>>> Content-Type: text/html; charset=UTF-8
>>> Set-Cookie: X-Auth-Token=AUTH_tk0f4473fa1234567890; Path=/
>>> X-Storage-Token: AUTH_tk0f4473fa1234567890
>>> Content-Length: 0
>>> X-Trans-Id: txcfefd5396a0f4f4694707-0053217efd
>>>
>>>
>>> And then you can send a GET to https://swift.example.com/info
>>>
>>> $> curl https://cloud.swiftstack.com/info
>>>
>>>
>>> One more information here, the /info was available since Swift 1.11.0
>>> https://github.com/openstack/swift/blob/master/CHANGELOG#L120-L125
>>>
>>>
>>>
>>>
>>>
>>> 2014-03-13 14:41 GMT+08:00 Ben McCann <ben at benmccann.com>:
>>>
>>> Thanks Samuel. That's super helpful. I didn't realize that SLO could be
>>>> turned on/off on the server. I thought it was merely a client-side
>>>> convention.
>>>>
>>>> swift stat doesn't return a StorageUrl for me. It just returns:
>>>>        Account:
>>>>     Containers:
>>>>        Objects:
>>>>          Bytes:
>>>>    Meta Cdn-Id:
>>>>    Meta Nas-Id:
>>>>    X-Timestamp:
>>>>     X-Trans-Id:
>>>>  Accept-Ranges:
>>>>
>>>> I'll see if I can find out from someone at SoftLayer whether they have
>>>> an old version or have large objects turned off unless you can think of any
>>>> other ways I might be able to find out by querying the server.
>>>>
>>>> Thanks,
>>>> Ben
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Mar 12, 2014 at 11:09 PM, Samuel Merritt <sam at swiftstack.com>wrote:
>>>>
>>>>> On 3/12/14 9:14 PM, Ben McCann wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm attempting to download a large file which I have uploaded to
>>>>>> SoftLayer Object Storage using the swift tool. It's downloading only
>>>>>> the
>>>>>> manifest and is not downloading the segments. Any tips on figuring out
>>>>>> what is going wrong?
>>>>>>
>>>>>
>>>>> It's entirely possible that the cluster in question lacks static large
>>>>> object support. Maybe they're running an old version of Swift
>>>>> (pre-1.8.0/Grizzly), or maybe they haven't enabled the static large object
>>>>> feature.
>>>>>
>>>>> What you'll need to do is probe the cluster to see if static large
>>>>> objects are enabled.
>>>>>
>>>>> The easy way is to politely ask the cluster with a GET request to
>>>>> /info. Run `swift stat -v`, then take the "StorageURL" field and change the
>>>>> path to be just "/info". Strip off any path that's already there.
>>>>>
>>>>> For example, if the output looks like
>>>>>
>>>>>     StorageURL: http://192.168.22.2:8080/v1/AUTH_test
>>>>>     Auth Token: AUTH_tk89454677b67348e09eefc1d6fe4dfa68
>>>>>        Account: AUTH_test
>>>>>     Containers: 1
>>>>>     [...]
>>>>>
>>>>> then you'd simply run
>>>>>
>>>>>     curl http://192.168.22.2:8080/info
>>>>>
>>>>> The response will be a JSON hash, so maybe run it through a JSON
>>>>> pretty-printer, e.g.
>>>>>
>>>>>     curl http://192.168.22.2:8080/info | python -mjson.tool
>>>>>
>>>>> On my development machine, the response looks something like this
>>>>> (some items removed for brevity):
>>>>>
>>>>> {
>>>>>     "container_quotas": {},
>>>>>     "formpost": {},
>>>>>     "slo": {                   <---- LOOK FOR THIS
>>>>>         "max_manifest_segments": 1000,
>>>>>         "max_manifest_size": 2097152,
>>>>>         "min_segment_size": 71
>>>>>     },
>>>>>     "swift": {
>>>>>         "account_listing_limit": 10000,
>>>>>         "container_listing_limit": 10000,
>>>>>         "max_account_name_length": 256,
>>>>>         "max_container_name_length": 256,
>>>>>         "max_file_size": 5368709122,
>>>>>         "max_meta_count": 90,
>>>>>         "max_meta_name_length": 128,
>>>>>         "max_meta_value_length": 256,
>>>>>         "max_object_name_length": 1024,
>>>>>         "version": "1.12.0.106.ga08e813"
>>>>>     }
>>>>> }
>>>>>
>>>>> If you don't see SLO in there, then you're out of luck. Bug Softlayer
>>>>> and see if they'll turn it on.
>>>>>
>>>>> Now, it's also possible that you'll get a 412 response with the body
>>>>> "Bad URL" when you run the curl command above. That will tell you that the
>>>>> cluster is running 1.11.0 or earlier, but it won't tell you anything about
>>>>> SLO support. In that case, try uploading a normal file (cat picture or
>>>>> something) with the header "X-Static-Large-Object: yes". If that succeeds,
>>>>> then SLO is disabled; if SLO is enabled, then the PUT will fail.
>>>>>
>>>>> A more verbose description is available here:
>>>>> https://ask.openstack.org/en/question/7005/static-large-objects/
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
>>>>> openstack
>>>>> Post to     : openstack at lists.openstack.org
>>>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
>>>>> openstack
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> about.me/benmccann
>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Mailing list:
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to     : openstack at lists.openstack.org
>>> Unsubscribe :
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>
>>>
>>
>>
>> --
>> about.me/benmccann
>>
>> _______________________________________________
>> Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>


-- 
about.me/benmccann
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140313/3407f22a/attachment.html>


More information about the Openstack mailing list