[Openstack] SWIFT - max file/object limits and tombstone question
Remo Mattei
Remo at italy1.com
Mon Aug 11 20:31:07 UTC 2014
You can use your swift-proxy to do that automatically.
Remo
On Aug 11, 2014, at 15:06, Brent Troge <brenttroge2016 at gmail.com> wrote:
> My CMS folks will not buy into adding more steps to their ingest process. Meaning they will not accept having to create the file segments and corresponding manifest. I will do more reading, hopefully there is some middleware that can make this whole process transparent.
>
> Thanks for your response!
>
>
> On Mon, Aug 11, 2014 at 12:56 PM, John Dickinson <me at not.mn> wrote:
> inline
>
> On Aug 11, 2014, at 10:38 AM, Brent Troge <brenttroge2016 at gmail.com> wrote:
>
> >
> > By default the maximum object size is 5G. Outside of increased replication times, would there be any impacts if I increase that value to 10G? The reason is I have to store upto 10G media files. Using the large file manifest just isnt going to work given Smooth Streaming or Adboe HDS delivery.
> >
>
> I'd like to understand more about why a large object manifest (especially the static large objects) won't work.
>
> But, to answer your question, increasing the max object size can affect the balance of drive fullness. Your drive fullness is directly related to the ratio of the max object size to the size of the drives. Swift places approximately equal numbers of objects on each drive (normalized to the drive weight). This works fine with with large numbers of objects because the hashing splays the object across all drives evenly. So, if you increase the max object size, everything will still work. But you'll need to keep a more careful eye on your capacity planning.
>
>
>
> > For Apple HLS delivery the media files are stored with a .ts extension. Would that cause any conflicts with tombstone files? Would Swfit mistakenly mark these media files as candiates for deletion? What would happen to a HLS .ts file once it needs to be marked as a tombstone file?
>
> The logical object name (eg "AUTH_foo/bar_container/some/object/name/here/that/is/long.ts") isn't part of the name used on the local media storage (ie the file on the hard drive). There is zero issue with using a ".ts" suffix for your object names.
>
>
> --John
>
>
>
>
> !DSPAM:1,53e9152f87271968467479! _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
> !DSPAM:1,53e9152f87271968467479!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140811/1db85bbd/attachment.html>
More information about the Openstack
mailing list