[Openstack] SWIFT - max file/object limits and tombstone question

Brent Troge brenttroge2016 at gmail.com
Mon Aug 11 19:06:21 UTC 2014

My CMS folks will not buy into adding more steps to their ingest process.
Meaning they will not accept having to  create the file segments and
corresponding manifest. I will do more reading, hopefully there is some
middleware that can make this whole process transparent.

Thanks for your response!

On Mon, Aug 11, 2014 at 12:56 PM, John Dickinson <me at not.mn> wrote:

> inline
> On Aug 11, 2014, at 10:38 AM, Brent Troge <brenttroge2016 at gmail.com>
> wrote:
> >
> > By default the maximum object size is 5G. Outside of increased
> replication times, would there be any impacts if I increase that value to
> 10G? The reason is I have to store upto 10G media files.  Using the large
> file manifest just isnt going to work given Smooth Streaming or Adboe HDS
> delivery.
> >
> I'd like to understand more about why a large object manifest (especially
> the static large objects) won't work.
> But, to answer your question, increasing the max object size can affect
> the balance of drive fullness. Your drive fullness is directly related to
> the ratio of the max object size to the size of the drives. Swift places
> approximately equal numbers of objects on each drive (normalized to the
> drive weight). This works fine with with large numbers of objects because
> the hashing splays the object across all drives evenly. So, if you increase
> the max object size, everything will still work. But you'll need to keep a
> more careful eye on your capacity planning.
> > For Apple HLS delivery the media files are stored with a .ts extension.
> Would that cause any conflicts with tombstone files? Would Swfit mistakenly
>  mark these media files as candiates for deletion? What would happen to a
> HLS .ts file once it needs to be marked as a tombstone file?
> The logical object name (eg
> "AUTH_foo/bar_container/some/object/name/here/that/is/long.ts") isn't part
> of the name used on the local media storage (ie the file on the hard
> drive). There is zero issue with using a ".ts" suffix for your object names.
> --John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140811/894cbf00/attachment.html>

More information about the Openstack mailing list