Open Stack

Hey Andi,

The backup implementation is up to the storage providers.  As for
lunr, we are working on a very similar feature set to back up volumes
to swift (and only backing up the changes since the last backup).

--
Chuck

On Thu, Jul 21, 2011 at 1:21 PM, andi abes <andi.abes at gmail.com> wrote:
> hmm - they definitely muddy the waters, but provide a really cool feature
> set:
>
> Amazon EBS Snapshots
>
> Amazon EBS provides the ability to back up point-in-time snapshots of your
> data to Amazon S3 for durable recovery. Amazon EBS snapshots are incremental
> backups, meaning that only the blocks on the device that have changed since
> your last snapshot will be saved. If you have a device with 100 GBs of data,
> but only 5 GBs of data has changed since your last snapshot, only the 5
> additional GBs of snapshot data will be stored back to Amazon S3. Even
> though the snapshots are saved incrementally, when you delete a snapshot,
> only the data not needed for any other snapshot is removed. So regardless of
> which prior snapshots have been deleted, all active snapshots will contain
> all the information needed to restore the volume. In addition, the time to
> restore the volume is the same for all snapshots, offering the restore time
> of full backups with the space savings of incremental
>
>
> That quoted - it's not exactly a low bar to meet in terms of capability.
> Chuck - are you proposing that as the target for Diablo?
>
> p.s - typing on a real keyboard is so much easier than an iPad, and leads to
> much better grammar...
> On Thu, Jul 21, 2011 at 12:19 PM, Chuck Thier <cthier at gmail.com> wrote:
>>
>> Hey Andi,
>>
>> Perhaps it would be better to re-frame the question.
>>
>> What should the base functionality of the Openstack API for
>> backup/snapshot functionality be?
>>
>> I'm looking at it from the perspective of initially providing the
>> capabilities that EC2/EBS currently provides (which they call
>> snapshots).  To me, this is the absolute base of what is needed, and
>> is what I am basically proposing as the idea of backups.
>>
>> I also see that allowing for true volume snapshot capabilities are
>> desirable down the road.  The difficulty with snapshots, is that their
>> properties can vary greatly between different storage systems, and
>> thus needs some care in defining what a Nova Volume snapshot should
>> support.  I would expect that the different storage providers would
>> initially provide this support through extensions to the API.  At that
>> point it may be easier to find what commonalities there are, and to
>> find what types of features are most demanded in the cloud.
>>
>> --
>> Chuck
>>
>> On Thu, Jul 21, 2011 at 5:57 AM, Andiabes <andi.abes at gmail.com> wrote:
>> > I think vish pointed out the main differences between the 2 entities,
>> > and maybe that can lead to name disambiguation...
>> >
>> > Backup is a full copy, and usable without the original object being
>> > available in any state ( original or modified). It's expensive, since it's a
>> > full copy. Main use cases are dr and recovery.
>> >
>> > Snapshot represents a point in time state of the object. It's relatively
>> > cheap ( with the expectation that some copy-on-write or differencing
>> > technique is used). Only usable if the reference point of the snapshot is
>> > available (could be thought of as an incremental backup); what that
>> > reference point is depends on the underlying implementation technology. Main
>> > use case is rewinding to so a historic state some time in the future.
>> >
>> > That said, with the prereqs met, both can probably be used to mount a
>> > new volume.
>> > Reasonable?
>> >
>> > On Jul 20, 2011, at 5:27 PM, Chuck Thier <cthier at gmail.com> wrote:
>> >
>> >> Yeah, I think you are illustrating how this generates much confusion :)
>> >>
>> >> To try to be more specific, the base functionality should be:
>> >>
>> >> 1. Create a point in time backup of a volume
>> >> 2. Create a new volume from a backup (I guess it seems reasonable to
>> >> call this a clone)
>> >>
>> >> This emulates the behavior of what EC2/EBS provide with volume
>> >> snapshots.  In this scenario, a "restore" is create a new volume from
>> >> the backup, and delete the old volume.
>> >>
>> >> In the Storage world, much more can generally be done with snapshots.
>> >> For example in most storage system snapshots are treated just like a
>> >> normal volume and can be mounted directly.  A snapshot is often used
>> >> when creating a backup to ensure that you have a consistent point in
>> >> time backup, which I think most of the confusion comes from.
>> >>
>> >> What we finally call it doesn't matter as much to me, as long as we
>> >> paint a consistent story that isn't confusing, and that we get it in
>> >> the Openstack API.
>> >>
>> >> --
>> >> Chuck
>> >>
>> >> On Wed, Jul 20, 2011 at 3:33 PM, Vishvananda Ishaya
>> >> <vishvananda at gmail.com> wrote:
>> >>> In rereading this i'm noticing that you are actually suggesting
>> >>> alternative usage:
>> >>>
>> >>> backup/clone
>> >>>
>> >>> snapshot/restore
>> >>>
>> >>> Correct?
>> >>>
>> >>> It seems like backup and snapshot are kind of interchangable.  This is
>> >>> quite confusing, perhaps we should refer to them as:
>> >>>
>> >>> partial-snapshot
>> >>>
>> >>> whole-snapshot
>> >>>
>> >>> or something along those lines that conveys that one is a differencing
>> >>> image and one is a copy of the entire object?
>> >>>
>> >>> On Jul 20, 2011, at 12:01 PM, Chuck Thier wrote:
>> >>>
>> >>>> At the last developers summit, it was noted by many, that the idea of
>> >>>> a volume snaphsot in the cloud is highly overloaded.  EBS uses the
>> >>>> notion of snapshots for making point in time backups of a volume that
>> >>>> can be used to create a new volume from.  These are not true
>> >>>> snapshots
>> >>>> though from a storage world view.  Because of this I would like to
>> >>>> make the following proposal:
>> >>>>
>> >>>> Add a backup API to the Openstack API for Nova Volume.  This is to
>> >>>> provide EBS style snapshot functionality in the Openstack API.  I'm
>> >>>> proposing to name it backup instead of snapshot as that seems to
>> >>>> better describe what is happening.  It also allows room for other
>> >>>> storage backends to expose real snapshot capabilities down the road.
>> >>>>
>> >>>> In the case of Lunr, we would be making backups of volumes to swift
>> >>>> (possibly abstracted through glance in the future).
>> >>>>
>> >>>> I have started a blueprint and spec at:
>> >>>>
>> >>>> https://blueprints.launchpad.net/nova/+spec/backups-api
>> >>>> http://etherpad.openstack.org/volume-backup
>> >>>>
>> >>>> Please feel free to comment and contribute.
>> >>>>
>> >>>> --
>> >>>> Chuck
>> >>>>
>> >>>> _______________________________________________
>> >>>> Mailing list: https://launchpad.net/~openstack
>> >>>> Post to     : openstack at lists.launchpad.net
>> >>>> Unsubscribe : https://launchpad.net/~openstack
>> >>>> More help   : https://help.launchpad.net/ListHelp
>> >>>
>> >>>
>> >>
>> >> _______________________________________________
>> >> Mailing list: https://launchpad.net/~openstack
>> >> Post to     : openstack at lists.launchpad.net
>> >> Unsubscribe : https://launchpad.net/~openstack
>> >> More help   : https://help.launchpad.net/ListHelp
>> >
>
>