Open Stack

Hello everyone,

I would like to ask a question. But, first of all, I would like to say that
I'm new to OpenStack so the question might be irrelevant. From what I've
understood, the idea is to back up an entire stack including VMs, volumes,
networks etc. Let's call the information about how these pieces are
interconnected - a topology. This topology also has to be backed up along
with VMs, volumes, networks, right? And then this topology can be used to
restore the entire stack. As for me, it looks very similar to what the Heat
project does. Am I right? So maybe it's possible to use the Heat project
for this kind of backup/restore functionality?

Best regards,
Alex




On Sun, Sep 1, 2013 at 10:23 PM, Giri Basava <giri.basava at triliodata.com>wrote:

> Dear All,
>
> This is a great discussion. If I understand this correctly, this is a
> proposal for data protection as a whole for the OpenStack cloud, however
> this is not yet an official "incubation" request. We are having a good
> discussion on how we can better serve the adoption of OpenStack.
>
> Having said that, the proposal will reuse the existing API and
> contributions by the community that are already in place. For example,
> Catlin's point is very valid... the Cinder storage vendor knows the best
> way to implement snapshots for their storage platforms. No doubt, Raksha
> should be leveraging that IP. Similarly Raksha will be leveraging Nova,
> Swift as well as Glance. Don't forget Neutron.... networking is very
> critical part of data protection for any VM or set of VMs.
>
> No one project has one single answer for a comprehensive data protection.
> The capabilities for backup and recovery exist in silos in various
> projects...
>
> 1. Images are backed-up by Nova
> 2. Volumes are backed-up by Cinder
> 3. I am not aware of a solution that can backup network configuration.
> 4. Not sure if we have something that can backup the resources of a VM (
> vCPUs, Memory Configuration etc.)
> 5. One can't schedule and automate the above very easily.
>
> Ronen's point about consistency groups is right on the mark. We need to
> treat an application as an unit that may span multiple VMs, one or more
> images and one or more volumes.
>
> Just to reiterate, some form of these capabilities exist in the current
> projects, however as a user of OpenStack, I may not have a simple one click
> solution.
>
> With this proposal, the ask is to engage in a discussion to address the
> above use cases. IMHO we are on the right track with these discussions. We
> will be submitting the code in few days and looking forward for your
> feedback. I would also suggest a design summit where we can flush out more
> details.
>
> Regards,
> Giri
>
> -----Original Message-----
> From: Avishay Traeger [mailto:AVISHAY at il.ibm.com]
> Sent: Saturday, August 31, 2013 10:53 PM
> To: OpenStack Development Mailing List
> Subject: Re: [openstack-dev] Proposal for Raksha, a Data Protection As a
> Service project
>
> +1
>
>
>
> From:   Ronen Kat/Haifa/IBM at IBMIL
> To:     OpenStack Development Mailing List
>             <openstack-dev at lists.openstack.org>,
> Date:   09/01/2013 08:02 AM
> Subject:        Re: [openstack-dev] Proposal for Raksha, a Data Protection
> As a
>             Service project
>
>
>
> Hi Murali,
>
> Thanks for answering. I think the issues you raised indeed make sense, and
> important ones.
>
> We need to provide backup both for:
> 1. Volumes
> 2. VM instances (VM image, VM metadata, and attached volumes)
>
> While the Cinder-backup handles (1), and is a very mature service, it does
> not provide (2), and for Cinder-backup it does not make sense to handle (2)
> as well.
> Backup of VMs (as a package) is beyond the scope of Cinder, which implies
> that indeed something beyond Cinder should take this task.
> I think this can be done by having Nova orchestrate or assist the backup,
> either of volumes or VMs.
>
> I think that from a backup perspective, there is also a need for
> "consistency groups" - the set of entities (volumes) that are considered a
> single logical unit and should be backup together.
> This logical consistency group could be larger than a VM, but a VM is a
> good starting point.
>
> In any case, we should adopt the "off-load" approach:
> 1. Handle application consistency issues using Nova as it manages the VMs.
> Add to Nova functionality to support live and consistent backup -
> including orchestrating volume backup using Cinder 2. Have Cinder do the
> volume "backup", and Cinder then can delegate the task to the
> Storage/hypervisor or anyone else who provide a backup driver
>
> While a new project is a neat package that addresses the issues, but does
> it worth the work?
> OpenStack projects are complex, and successful projects require a lot of
> work and long-term maintenance, which is the real pain for open source
> projects, as the team tend to be very dynamic.
>
> My two cents is to adopt the "nova-networking" and "nova-volume" approach,
> try to extend the current work within Nova and Cinder, and if we find out
> it does not make sense anymore, explain the issues, and split the work to a
> new project.
> This way it, if a backup project is indeed needed, you already have the
> community to support the effort, and you already have a mature solution.
>
> Regards,
> __________________________________________
> Ronen I. Kat
> Storage Research
> IBM Research - Haifa
> Phone: +972.3.7689493
> Email: ronenkat at il.ibm.com
>
>
>
>
> From:            Caitlin Bestler <caitlin.bestler at nexenta.com>
> To:              Murali Balcha <Murali.Balcha at triliodata.com>,
> Cc:              OpenStack Development Mailing List
>             <openstack-dev at lists.openstack.org>
> Date:            31/08/2013 07:25 AM
> Subject:                 Re: [openstack-dev] Proposal for Raksha, a Data
> Protection As a
>             Service project
>
>
>
> On 8/30/2013 12:49 PM, Murali Balcha wrote:
> > Hi Caitlin,
> > Did you get a chance to look at the wiki? It describes the raksha
> functionality in detail.
>  > It includes more than volume backup. It includes vm images, all  >
> volumes and network configurations associated with vms and it  > supports
> incremental backups too. Volume backup is essential for  > implementing
> backup solution but not necessarily sufficient.
>
> Cinder already allows backing volumes up to Swift, and in fact allows
> incremental backups.
>
> Any code you write will not back up a vendor's volume more efficiently
> than the vendor's code itself can.
>
> The vendor's knowledge of how the data is stored is probably sufficient,
> but in this case a vendor has a far more powerful advantage. The vendor can
> transfer the volume directly to the Swift server. Your service, since it
> running on a compute node rather than the vendor's box, will first have to
> fetch the content and *then* send it to Swift.
>
> That's twice as much network traffic. This is not trivial when volumes are
> big, which they tend to be.
>
> If this service is implemented, customer who are using vendor backends
> such as NexentaStor, Netapp or CEPH will see their performance drop.
> That will clearly be unacceptable. New featqures are not allowed to trash
> existing performance, especially when they are not actually providing any
> new service to customers who already have volume backends with these
> features.
>
> You would need to have a proposal to work with the existing Cinder backend
> Volume Drivers that in no way removed any option vendors have currently to
> optimize performance.
>
> Doing that in a new project, rather than within Cinder, can only make life
> harder on the vendors and discourage participation in OpenStack.
>
> I believe all of the features you are looking at can be accomodated by
> taskflows using the existing Volume Driver feature (as evolving) in Cinder.
>  A new project is not justified, and it will risk creating a major
> performance regression for some customers.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130901/2c12cc84/attachment.html>