Open Stack

Hello Ross,

On 05.05.15 21:54, Ross Lillie wrote:
> My understanding is that Swift should automagically split files greater
> that 5G into multiple segments grouped under a metafile but this appears
> to not be working. This was working under the Havana release (Ubuntu)
> using the Swift File System jar file downloaded from the Marantis web
> site.  All current testing is based up the Juno release and when
> performing a distcp using the openstack-hadoop jar file shipped as part
> of the latest hadoop distros.

I don't know the client you're using, but Swift itself (on the server
side) never splits data into segments on its own, and never did so.
Currently it's up to the client to ensure data is broken down into
segments of max. 5GB size.

There were some ideas in the past to implement this feature, however
this raised different problems. Have a look at the history of large
objects:

http://docs.openstack.org/developer/swift/overview_large_objects.html#history

I would assume that there was a change on the client side implementation
that changed the behavior for you?

Best Regards,

Christian