Open Stack

There was a recent post on a SWIFT tape implementation at https://github.com/BDT-GER/SWIFT-TLC

Tim

> -----Original Message-----
> From: Samuel Merritt [mailto:sam at swiftstack.com]
> Sent: 06 May 2015 19:03
> To: openstack at lists.openstack.org
> Subject: Re: [Openstack] Swift - Adding S3 Glacier like interface in Swift & Swift3
> Object Storage
> 
> On 5/6/15 2:34 AM, Bala wrote:
> > I am new to this list so please excuse me if I posted it in wrong list.
> >
> > We have a tape library which we would like to integrate with OpenStack
> > Swift & Swift3 object storage service to provide S3 interface.
> >
> > The current file system we have for the library has been integrated
> > with Swift storage service and manages changer robot & tapes.
> >
> > This works well for writing.
> >
> > However for reading, loading a tape takes longer when GET requests are
> > received, in some cases over 5 minutes and this causes timeout error.
> > Most of the data stored in these tapes are archival data. This get
> > worsen when multiple GET requests received (muti-user) for objects
> > which are stored in different tapes.
> >
> > Due to the longer read times, we are looking to provide Amazon S3
> > Glacier like interface through Swift & Swift3 so that clients can
> > issue a POST OBJECT RESTORE request and wait for the data to be moved
> > to temporary store/cache.
> >
> > I have come across a similar request
> >
> > http://openstack-dev.openstack.narkive.com/kI72vk9l/ltfs-integration-w
> > ith-openstack-swift-for-scenario-like-data-archival-as-a-service
> >
> > and understand the suggestions.
> >
> > We would like to provide S3 Glacier like interface than Swift Storage
> > policies if we can.
> >
> > I would be great full if you could kindly advise
> >
> > 1. How hard is to change Swift & Swift3 code base to provide S3
> > Glacier like interface
> 
> It's not easy, that's for sure. Swift's API is all synchronous: issue a GET, receive
> the object; issue a PUT, create an object; et cetera.
> Glacier-style asynchronous retrieval is something completely new and different.
> 
> Some food for thought: where will you store pending retrieval requests?
> How will you ensure that retrieval requests survive disk and machine failures the
> way everything else in Swift does?
> 
> I'm not asking you to answer here (though you can if you want to, of course);
> I'm just trying to nudge your thoughts in the right direction.
> 
> > 2. Can this be done through Swift storage policies alone.
> 
> No. A storage policy determines where and how its objects are stored, which
> affects things like access times and storage cost. The API for accessing those
> objects does not change based on the storage policy.
> 
> > 3. Do we have to modify Swift Auditor service to do a tape based
> > checking rather than object based.
> 
> You mean audit in order? Probably a good idea, otherwise your tapes will spend
> all day seeking.
> 
> > 4. Would Swift replication service cause frequent Tape change request.
> 
> I'd guess that it would, but nobody knows for sure. As far as I know, nobody has
> jammed tape-library support into Swift before. You're the first. Report back and
> let everyone know how it goes. :)
> 
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack