Open Stack

I am not an expert in Heat but the way I understood the Heat project is that it is an orchestration layer that instantiate a composite application based on a template definition. The template may identify the vms, networks and storage resources needed to instantiate the composite application. Using the template, a tenant may instantiate one of more instances of composite application. 

Backup service such as Raksha has to backup composite application instances in their entirety. It can look into Heat meta data to understand the application definition and correctly backup the application.

 I am not sure Heat can manage individual application instances once they are created. I need to do more research on Heat, but I also defer comments to Heat experts.

Other way to integrate Heat with Raskha is to make a backup policy part of Heat template and let Heat setup correct backup policy by calling Rakha Apis during composite application instantiation.

Thanks,
Murali Balcha

On Sep 2, 2013, at 7:13 AM, "Zane Bitter" <zbitter at redhat.com> wrote:

> On 01/09/13 23:11, Alex Rudenko wrote:
>> Hello everyone,
>> 
>> I would like to ask a question. But, first of all, I would like to say
>> that I'm new to OpenStack so the question might be irrelevant. From what
>> I've understood, the idea is to back up an entire stack including VMs,
>> volumes, networks etc. Let's call the information about how these pieces
>> are interconnected - a topology. This topology also has to be backed up
>> along with VMs, volumes, networks, right? And then this topology can be
>> used to restore the entire stack. As for me, it looks very similar to
>> what the Heat project does. Am I right? So maybe it's possible to use
>> the Heat project for this kind of backup/restore functionality?
>> 
>> Best regards,
>> Alex
> 
> That's actually an excellent question.
> 
> One of the things that's new in Heat for the Havana release is Suspend/Resume operations on stacks. Basically this involves going through the stack in (reverse) dependency order and calling suspend/resume APIs for each resource where that makes sense. Steve Hardy has written the code for this in such a way as to be pretty generic and allow us to add more operations quite easily in the future.
> 
> So to the extent that you just require something to go through every resource in a stack in dependency order and call an *existing* backup API, then Heat could fit the bill. If you require co-ordination between e.g. Nova and Cinder then Heat is probably not a good vehicle for implementing that.
> 
> cheers,
> Zane.
> 
> 
>> On Sun, Sep 1, 2013 at 10:23 PM, Giri Basava <giri.basava at triliodata.com
>> <mailto:giri.basava at triliodata.com>> wrote:
>> 
>>    Dear All,
>> 
>>    This is a great discussion. If I understand this correctly, this is
>>    a proposal for data protection as a whole for the OpenStack cloud,
>>    however this is not yet an official "incubation" request. We are
>>    having a good discussion on how we can better serve the adoption of
>>    OpenStack.
>> 
>>    Having said that, the proposal will reuse the existing API and
>>    contributions by the community that are already in place. For
>>    example, Catlin's point is very valid... the Cinder storage vendor
>>    knows the best way to implement snapshots for their storage
>>    platforms. No doubt, Raksha should be leveraging that IP. Similarly
>>    Raksha will be leveraging Nova, Swift as well as Glance. Don't
>>    forget Neutron.... networking is very critical part of data
>>    protection for any VM or set of VMs.
>> 
>>    No one project has one single answer for a comprehensive data
>>    protection. The capabilities for backup and recovery exist in silos
>>    in various projects...
>> 
>>    1. Images are backed-up by Nova
>>    2. Volumes are backed-up by Cinder
>>    3. I am not aware of a solution that can backup network configuration.
>>    4. Not sure if we have something that can backup the resources of a
>>    VM ( vCPUs, Memory Configuration etc.)
>>    5. One can't schedule and automate the above very easily.
>> 
>>    Ronen's point about consistency groups is right on the mark. We need
>>    to treat an application as an unit that may span multiple VMs, one
>>    or more images and one or more volumes.
>> 
>>    Just to reiterate, some form of these capabilities exist in the
>>    current projects, however as a user of OpenStack, I may not have a
>>    simple one click solution.
>> 
>>    With this proposal, the ask is to engage in a discussion to address
>>    the above use cases. IMHO we are on the right track with these
>>    discussions. We will be submitting the code in few days and looking
>>    forward for your feedback. I would also suggest a design summit
>>    where we can flush out more details.
>> 
>>    Regards,
>>    Giri
>> 
>>    -----Original Message-----
>>    From: Avishay Traeger [mailto:AVISHAY at il.ibm.com
>>    <mailto:AVISHAY at il.ibm.com>]
>>    Sent: Saturday, August 31, 2013 10:53 PM
>>    To: OpenStack Development Mailing List
>>    Subject: Re: [openstack-dev] Proposal for Raksha, a Data Protection
>>    As a Service project
>> 
>>    +1
>> 
>> 
>> 
>>    From:   Ronen Kat/Haifa/IBM at IBMIL
>>    To:     OpenStack Development Mailing List
>>                 <openstack-dev at lists.openstack.org
>>    <mailto:openstack-dev at lists.openstack.org>>,
>>    Date:   09/01/2013 08:02 AM
>>    Subject:        Re: [openstack-dev] Proposal for Raksha, a Data
>>    Protection As a
>>                 Service project
>> 
>> 
>> 
>>    Hi Murali,
>> 
>>    Thanks for answering. I think the issues you raised indeed make
>>    sense, and important ones.
>> 
>>    We need to provide backup both for:
>>    1. Volumes
>>    2. VM instances (VM image, VM metadata, and attached volumes)
>> 
>>    While the Cinder-backup handles (1), and is a very mature service,
>>    it does not provide (2), and for Cinder-backup it does not make
>>    sense to handle (2) as well.
>>    Backup of VMs (as a package) is beyond the scope of Cinder, which
>>    implies that indeed something beyond Cinder should take this task.
>>    I think this can be done by having Nova orchestrate or assist the
>>    backup, either of volumes or VMs.
>> 
>>    I think that from a backup perspective, there is also a need for
>>    "consistency groups" - the set of entities (volumes) that are
>>    considered a single logical unit and should be backup together.
>>    This logical consistency group could be larger than a VM, but a VM
>>    is a good starting point.
>> 
>>    In any case, we should adopt the "off-load" approach:
>>    1. Handle application consistency issues using Nova as it manages
>>    the VMs.
>>    Add to Nova functionality to support live and consistent backup -
>>    including orchestrating volume backup using Cinder 2. Have Cinder do
>>    the volume "backup", and Cinder then can delegate the task to the
>>    Storage/hypervisor or anyone else who provide a backup driver
>> 
>>    While a new project is a neat package that addresses the issues, but
>>    does it worth the work?
>>    OpenStack projects are complex, and successful projects require a
>>    lot of work and long-term maintenance, which is the real pain for
>>    open source projects, as the team tend to be very dynamic.
>> 
>>    My two cents is to adopt the "nova-networking" and "nova-volume"
>>    approach, try to extend the current work within Nova and Cinder, and
>>    if we find out it does not make sense anymore, explain the issues,
>>    and split the work to a new project.
>>    This way it, if a backup project is indeed needed, you already have
>>    the community to support the effort, and you already have a mature
>>    solution.
>> 
>>    Regards,
>>    __________________________________________
>>    Ronen I. Kat
>>    Storage Research
>>    IBM Research - Haifa
>>    Phone: +972.3.7689493 <tel:%2B972.3.7689493>
>>    Email: ronenkat at il.ibm.com <mailto:ronenkat at il.ibm.com>
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev