[Openstack] SWIFT - max file/object limits and tombstone question

John Dickinson me at not.mn
Mon Aug 11 17:56:31 UTC 2014


On Aug 11, 2014, at 10:38 AM, Brent Troge <brenttroge2016 at gmail.com> wrote:

> By default the maximum object size is 5G. Outside of increased replication times, would there be any impacts if I increase that value to 10G? The reason is I have to store upto 10G media files.  Using the large file manifest just isnt going to work given Smooth Streaming or Adboe HDS delivery.

I'd like to understand more about why a large object manifest (especially the static large objects) won't work.

But, to answer your question, increasing the max object size can affect the balance of drive fullness. Your drive fullness is directly related to the ratio of the max object size to the size of the drives. Swift places approximately equal numbers of objects on each drive (normalized to the drive weight). This works fine with with large numbers of objects because the hashing splays the object across all drives evenly. So, if you increase the max object size, everything will still work. But you'll need to keep a more careful eye on your capacity planning.

> For Apple HLS delivery the media files are stored with a .ts extension. Would that cause any conflicts with tombstone files? Would Swfit mistakenly  mark these media files as candiates for deletion? What would happen to a HLS .ts file once it needs to be marked as a tombstone file?

The logical object name (eg "AUTH_foo/bar_container/some/object/name/here/that/is/long.ts") isn't part of the name used on the local media storage (ie the file on the hard drive). There is zero issue with using a ".ts" suffix for your object names.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140811/c8b4d2b6/attachment.sig>

More information about the Openstack mailing list