Open Stack

I also believe (2) is the most workable option.

Full disclosure ... my current job is at EMC, and we just shipped a backup
product for the VMware vCloud (the one used for VMware vCloud Air -
http://vcloud.vmware.com/http://vcloud.vmware.com/).

First release of that project was wrapping up, and I was asked to look at
backup for OpenStack. As I am familiar with both open source community, and
with how to do backup at cloud scale ... the problem is an easy fit.

Different backup vendors might approach the problem in differing ways,
which calls for a pluggable backend.

What I find is that OpenStack is missing the hooks we need to do backup
efficiently. There are promising proposed bits, but ... we are not there
yet.

Storage for backups is of a different character than existing services in
OpenStack. Backup vendors need a place to plug in, and need some relevant
primitive operations.

Turns out the AWS folk already have rather nice support for backup from a
cloud developer's perspective. At present, there is nothing equivalent in
OpenStack. From a cloud developer's perspective, that is a huge lack.

While AWS is an API wrapped around a single service, OpenStack is an API
wrapped around differing services. This is both harder (in terms of
defining the API), and an advantage to customers with differing
requirements.

What lacks and is needed by backup vendors is the right set of primitives.




On Sun, Aug 31, 2014 at 8:19 AM, laserjetyang <laserjetyang at gmail.com>
wrote:

> I tend to say 2) is the best option. There are many open source or
> commercial backup software, and both for VMs and volume.
> If we do option 1), it reminds me to implement something similar to VMware
> method, and it will cause nova really heavy.
>
>
> On Sun, Aug 31, 2014 at 4:04 AM, Preston L. Bannister <
> preston at bannister.us> wrote:
>
>> You are thinking of written-for-cloud applications. For those the state
>> should not persist with the instance.
>>
>> There are a very large number of existing applications, not written to
>> the cloud model, but which could be deployed in a cloud. Those applications
>> are not all going to get re-written (as the cost is often greater than the
>> benefit). Those applications need some ready and efficient means of backup.
>>
>> The benefits of the cloud-application model and the cloud-deployment
>> model are distinct.
>>
>> The existing nova backup (if it worked) is an inefficient snapshot. Not
>> really useful at scale.
>>
>> There are two basic paths forward, here.  1) Build a complete common
>> backup implementation that everyone can use. Or 2) define a common API for
>> invoking backup, allow vendors to supply differing implementations, and add
>> to OpenStack the APIs needed by backup implementations.
>>
>> Given past history, there does not seem to be enough focus or resources
>> to get (1) done.
>>
>> That makes (2) much more likely. Reasonably sure we can find the interest
>> and resources for this path. :)
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 29, 2014 at 10:55 PM, laserjetyang <laserjetyang at gmail.com>
>> wrote:
>>
>>> I think the purpose of nova VM is not for persistent usage, and it
>>> should be used for stateless. However, there are use cases to use VM to
>>> replace bare metal applications, and it requires the same coverage, which I
>>> think VMware did pretty well.
>>> The nova backup is snapshot indeed, so it should be re-implemented to be
>>> fitting into the real backup solution.
>>>
>>>
>>> On Sat, Aug 30, 2014 at 1:14 PM, Preston L. Bannister <
>>> preston at bannister.us> wrote:
>>>
>>>> The current "backup" APIs in OpenStack do not really make sense (and
>>>> apparently do not work ... which perhaps says something about usage and
>>>> usability). So in that sense, they could be removed.
>>>>
>>>> Wrote out a bit as to what is needed:
>>>>
>>>> http://bannister.us/weblog/2014/08/21/cloud-application-backup-and-openstack/
>>>>
>>>> At the same time, to do efficient backup at cloud scale, OpenStack is
>>>> missing a few primitives needed for backup. We need to be able to quiesce
>>>> instances, and collect changed-block lists, across hypervisors and
>>>> filesystems. There is some relevant work in this area - for example:
>>>>
>>>> https://wiki.openstack.org/wiki/Nova/InstanceLevelSnapshots
>>>>
>>>> Switching hats - as a cloud developer, on AWS there is excellent
>>>> current means of backup-through-snapshots, which is very quick and is
>>>> charged based on changed-blocks. (The performance and cost both reflect use
>>>> of changed-block tracking underneath.)
>>>>
>>>> If OpenStack completely lacks any equivalent API, then the platform is
>>>> less competitive.
>>>>
>>>> Are you thinking about backup as performed by the cloud infrastructure
>>>> folk, or as a service used by cloud developers in deployed applications?
>>>> The first might do behind-the-scenes backups. The second needs an API.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 29, 2014 at 11:16 AM, Jay Pipes <jaypipes at gmail.com> wrote:
>>>>
>>>>> On 08/29/2014 02:48 AM, Preston L. Bannister wrote:
>>>>>
>>>>>> Looking to put a proper implementation of instance backup into
>>>>>> OpenStack. Started by writing a simple set of baseline tests and
>>>>>> running
>>>>>> against the stable/icehouse branch. They failed!
>>>>>>
>>>>>> https://github.com/dreadedhill-work/openstack-backup-scripts
>>>>>>
>>>>>> Scripts and configuration are in the above. Simple tests.
>>>>>>
>>>>>> At first I assumed there was a configuration error in my Devstack ...
>>>>>> but at this point I believe the errors are in fact in OpenStack.
>>>>>> (Also I
>>>>>> have rather more colorful things to say about what is and is not
>>>>>> logged.)
>>>>>>
>>>>>> Try to backup bootable Cinder volumes attached to instances ... and
>>>>>> all
>>>>>> fail. Try to backup instances booted from images, and all-but-one fail
>>>>>> (without logged errors, so far as I see).
>>>>>>
>>>>>> Was concerned about preserving existing behaviour (as I am currently
>>>>>> hacking the Nova backup API), but ... if the existing is badly broken,
>>>>>> this may not be a concern. (Makes my job a bit simpler.)
>>>>>>
>>>>>> If someone is using "nova backup" successfully (more than one backup
>>>>>> at
>>>>>> a time), I *would* rather like to know!
>>>>>>
>>>>>> Anyone with different experience?
>>>>>>
>>>>>
>>>>> IMO, the create_backup API extension should be removed from the
>>>>> Compute API. It's completely unnecessary and backups should be the purview
>>>>> of external (to Nova) scripts or configuration management modules. This API
>>>>> extension is essentially trying to be a Cloud Cron, which is inappropriate
>>>>> for the Compute API, IMO.
>>>>>
>>>>> -jay
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140831/1f453df3/attachment.html>