Open Stack

On 01/09/13 23:11, Alex Rudenko wrote:
> Hello everyone,
>
> I would like to ask a question. But, first of all, I would like to say
> that I'm new to OpenStack so the question might be irrelevant. From what
> I've understood, the idea is to back up an entire stack including VMs,
> volumes, networks etc. Let's call the information about how these pieces
> are interconnected - a topology. This topology also has to be backed up
> along with VMs, volumes, networks, right? And then this topology can be
> used to restore the entire stack. As for me, it looks very similar to
> what the Heat project does. Am I right? So maybe it's possible to use
> the Heat project for this kind of backup/restore functionality?
>
> Best regards,
> Alex

That's actually an excellent question.

One of the things that's new in Heat for the Havana release is 
Suspend/Resume operations on stacks. Basically this involves going 
through the stack in (reverse) dependency order and calling 
suspend/resume APIs for each resource where that makes sense. Steve 
Hardy has written the code for this in such a way as to be pretty 
generic and allow us to add more operations quite easily in the future.

So to the extent that you just require something to go through every 
resource in a stack in dependency order and call an *existing* backup 
API, then Heat could fit the bill. If you require co-ordination between 
e.g. Nova and Cinder then Heat is probably not a good vehicle for 
implementing that.

cheers,
Zane.


> On Sun, Sep 1, 2013 at 10:23 PM, Giri Basava <giri.basava at triliodata.com
> <mailto:giri.basava at triliodata.com>> wrote:
>
>     Dear All,
>
>     This is a great discussion. If I understand this correctly, this is
>     a proposal for data protection as a whole for the OpenStack cloud,
>     however this is not yet an official "incubation" request. We are
>     having a good discussion on how we can better serve the adoption of
>     OpenStack.
>
>     Having said that, the proposal will reuse the existing API and
>     contributions by the community that are already in place. For
>     example, Catlin's point is very valid... the Cinder storage vendor
>     knows the best way to implement snapshots for their storage
>     platforms. No doubt, Raksha should be leveraging that IP. Similarly
>     Raksha will be leveraging Nova, Swift as well as Glance. Don't
>     forget Neutron.... networking is very critical part of data
>     protection for any VM or set of VMs.
>
>     No one project has one single answer for a comprehensive data
>     protection. The capabilities for backup and recovery exist in silos
>     in various projects...
>
>     1. Images are backed-up by Nova
>     2. Volumes are backed-up by Cinder
>     3. I am not aware of a solution that can backup network configuration.
>     4. Not sure if we have something that can backup the resources of a
>     VM ( vCPUs, Memory Configuration etc.)
>     5. One can't schedule and automate the above very easily.
>
>     Ronen's point about consistency groups is right on the mark. We need
>     to treat an application as an unit that may span multiple VMs, one
>     or more images and one or more volumes.
>
>     Just to reiterate, some form of these capabilities exist in the
>     current projects, however as a user of OpenStack, I may not have a
>     simple one click solution.
>
>     With this proposal, the ask is to engage in a discussion to address
>     the above use cases. IMHO we are on the right track with these
>     discussions. We will be submitting the code in few days and looking
>     forward for your feedback. I would also suggest a design summit
>     where we can flush out more details.
>
>     Regards,
>     Giri
>
>     -----Original Message-----
>     From: Avishay Traeger [mailto:AVISHAY at il.ibm.com
>     <mailto:AVISHAY at il.ibm.com>]
>     Sent: Saturday, August 31, 2013 10:53 PM
>     To: OpenStack Development Mailing List
>     Subject: Re: [openstack-dev] Proposal for Raksha, a Data Protection
>     As a Service project
>
>     +1
>
>
>
>     From:   Ronen Kat/Haifa/IBM at IBMIL
>     To:     OpenStack Development Mailing List
>                  <openstack-dev at lists.openstack.org
>     <mailto:openstack-dev at lists.openstack.org>>,
>     Date:   09/01/2013 08:02 AM
>     Subject:        Re: [openstack-dev] Proposal for Raksha, a Data
>     Protection As a
>                  Service project
>
>
>
>     Hi Murali,
>
>     Thanks for answering. I think the issues you raised indeed make
>     sense, and important ones.
>
>     We need to provide backup both for:
>     1. Volumes
>     2. VM instances (VM image, VM metadata, and attached volumes)
>
>     While the Cinder-backup handles (1), and is a very mature service,
>     it does not provide (2), and for Cinder-backup it does not make
>     sense to handle (2) as well.
>     Backup of VMs (as a package) is beyond the scope of Cinder, which
>     implies that indeed something beyond Cinder should take this task.
>     I think this can be done by having Nova orchestrate or assist the
>     backup, either of volumes or VMs.
>
>     I think that from a backup perspective, there is also a need for
>     "consistency groups" - the set of entities (volumes) that are
>     considered a single logical unit and should be backup together.
>     This logical consistency group could be larger than a VM, but a VM
>     is a good starting point.
>
>     In any case, we should adopt the "off-load" approach:
>     1. Handle application consistency issues using Nova as it manages
>     the VMs.
>     Add to Nova functionality to support live and consistent backup -
>     including orchestrating volume backup using Cinder 2. Have Cinder do
>     the volume "backup", and Cinder then can delegate the task to the
>     Storage/hypervisor or anyone else who provide a backup driver
>
>     While a new project is a neat package that addresses the issues, but
>     does it worth the work?
>     OpenStack projects are complex, and successful projects require a
>     lot of work and long-term maintenance, which is the real pain for
>     open source projects, as the team tend to be very dynamic.
>
>     My two cents is to adopt the "nova-networking" and "nova-volume"
>     approach, try to extend the current work within Nova and Cinder, and
>     if we find out it does not make sense anymore, explain the issues,
>     and split the work to a new project.
>     This way it, if a backup project is indeed needed, you already have
>     the community to support the effort, and you already have a mature
>     solution.
>
>     Regards,
>     __________________________________________
>     Ronen I. Kat
>     Storage Research
>     IBM Research - Haifa
>     Phone: +972.3.7689493 <tel:%2B972.3.7689493>
>     Email: ronenkat at il.ibm.com <mailto:ronenkat at il.ibm.com>
>