Open Stack

There were some discussions over the past years.

I raised the question of Swift tape support in my keynote in Boston in 2011 (http://www.slideshare.net/noggin143/cern-user-story) but there was limited interest. LTFS makes it more likely but we should not underestimate the challenges. Ensuring bulk recall/migration (mounting tapes takes minutes), inventory catalogs to find the right tape and robotics (multiple interfaces to ask for a tape to be mounted) means it is not just a POSIX support question.

There was a blog in 2012 regarding a Glacier competitor (http://www.buildcloudstorage.com/2012/08/cold-storage-using-openstack-swift-vs.html) but I don't think things have progressed much beyond that.

It would need to be tiered (i.e. migrate whole collections rather than files) and a local catalog would be needed to map containers to tapes. Timeouts would be an issue since we are often waiting hours for recall (to ensure that multiple recalls for the same tape are grouped).

It is not an insolvable problem but it is not just a 'use LTFS' answer.

Tim

On 14 Nov 2014, at 19:06, Samuel Merritt <sam at swiftstack.com<mailto:sam at swiftstack.com>> wrote:

On 11/13/14, 10:19 PM, Sachin Goswami wrote:
In OpenStack Swift - xfs file system is integrated which provides a
maximum file system size of 8 exbibytes minus one byte (263-1 bytes).

Not exactly. The Swift storage nodes keep their data on POSIX filesystems with support for extended attributes. While XFS filesystems are typically used, XFS is not required.

We are studying use of LTFS integration with OpenStack Swift for
scenario like - *Data Archival as a Service* .

Was integration of LTFS with Swift considered before? If so, can you
please share your study output? Will integration of LTFS with Swift
fit into existing Swift architecture ?

Assuming it's POSIX enough and supports extended attributes, a tape filesystem on a spinning disk might technically work, but I don't see it performing well at all.

If you're talking about using actual tapes for data storage, I can't imagine that working out for you. Most clients aren't prepared to wait multiple minutes for HTTP responses while a tape laboriously spins back and forth, so they'll just time out.


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141114/7b1349ee/attachment.html>