<div dir="ltr">Hi Fausto,<br><br>To be clear, I am not in any way critical of Freezer and the folk putting in work. (Please, I want to be <i>entirely</i> clear on this point! Also, saw your presentation in Vancouver.)<br><br>That said, Freezer is a bit of a Swiss-Army-Knife set of combined backup functions. Sometimes it is better to focus on a single aspect (or few). The new features landing in QEMU present an opportunity. A project focused solely on that opportunity, to work through initial set of issues, makes a lot of sense to me. (Something like how KVM forked QEMU for a time, to build faster x86 emulation.) <br><br>I do not see these as competing projects, and more as cooperative. The Ekko work should be able to plug into Freezer, cleanly. <br><br>Aspects of the problem, as I see:<div><ol><li>Extracting efficient instance backups from OpenStack (via new QEMU function).</li><li>Storing backups in efficient form (general implementation, and vendor-specific supersets).<br></li><li>Offering an OpenStack backup-service API, with core and vendor-specific extensions.</li></ol><div><br></div><div>Vendors (like my employer, EMC) might be somewhat opinionated about (2), and for reason. :)<br><br>The huge missing piece is (1), and a focused project seems to make a lot of sense.</div><div><br></div><div>As to (3), that looks like a good topic for further discussion. :)</div><div><br></div><br>My $.02.<br><br><br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jan 30, 2016 at 5:36 PM, Fausto Marzi <span dir="ltr"><<a href="mailto:fausto.marzi@gmail.com" target="_blank">fausto.marzi@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div>Hi Preston,<br></div>No need to apologize. They are aspect of the same problem. <br>However, VMs backup is one of the many aspects that we are approaching here, such as:<br><br></div>- VM backups<br></div>- Volumes backups<br></div>- Specific applications consistent data backup (i.e. MySQL, Mongo, file system, etc)<br></div>- Provide capabilities to restore data even if keystone and swift are not available<br></div>- Upload data during backup to multiple media storage in parallel<br></div>- Web Interface<br></div>- Provide capability to synchronize backups for sharded data on multiple nodes<br></div>- Encryption<br></div>- File based incremental<br></div>- Block based incremental<br></div>- Tenant related data backup and restore<br></div><div>- Multi platform OS support (i.e. Linux, BSD, OSX, Windows, iOS, Android, etc)<br></div><div>- Everything is upstreamed.<br></div><div><br></div>This looks like a list of features... and actually it is.<br><br>Block based and some multi platform OS aside, all the mentioned features are provided to the date. Most of them are available since Kilo.<br><br></div>I agree with the multi API, room for vendors and to provide different approaches, but please let me say something (<b>not referring specifically to you or Sam or anyone</b>)<br><br>All the time people say you have to do this and that, but the facts are that at the end of the day, always the same 6 engineers (not even full time) are working on it since 2 years, investing professional and personal time on it.<br><br>We try to be open, to accept everybody (even the most arrogant), to implement features for whoever needs it, but the facts are that the only Companies that invested on it are HP, a bit Ericsson and Orange (apologize if I forgot anyone). We never said no to anyone about anything, never focused only to a single Company influence, never blocked a thing... and never will.<br><br></div>Wouldn't be better to join efforts if companies need a backup solution and have their own requirements implemented by a common public Team, rather then start creating many tools to solve the same set of problems? How can ever competition benefit this? How can ever fragmenting projects help to provide a better solution?<br><div><div><div><br></div><div>I'm sorry, but unless the TC or many people from the Community, tell us to do something different (in that case we'll do it straight away), we'll keep doing what we are doing, focusing on delivering what we think is the most advanced solution, according the resources and time we have.<br></div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><br></div><div>We need to understand that here the most important thing is to work in Team, to provide great tools to the Community, rather then thinking to be PTL or maintain independence just for the sake of it or focusing only on what's the best for a single Company. If this vision is not shared, then, unfortunately, good luck competing, while if the vision is shared... let's do together unprecedented things.<br><br></div><div>Many thanks,<br></div><div>Fausto<br></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Sun, Jan 31, 2016 at 1:01 AM, Preston L. Bannister <span dir="ltr"><<a href="mailto:preston@bannister.us" target="_blank">preston@bannister.us</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr">Seems to me there are three threads here.<div><br></div><div>The Freezer folk were given a task, and did the best possible to support backup given what OpenStack allowed. To date, OpenStack is simply not very good at supporting backup as a service. (Apologies to the Freezer folk if I misinterpreted.)<br><br>The patches (finally) landing in QEMU in support of incremental backup could be the basis for efficient backup services in OpenStack. This is all fairly high risk, in the short term. The bits that landed in QEMU 2.4 may not be sufficient (there are more QEMU patches trying to land). When put into production, we may find faults. For use in OpenStack, we may need changes in libvirt, and/or in Nova. (Or <i>maybe</i> not if usage for backup proves orthogonal.) The only way to work out the prior is to start. The timeline could be months or years.</div><div><br></div><div>There is a need for a common API for backup as a service in the cloud. Something more than imitating AWS. Allow some room for vendors with differing approach.</div><div><br></div><div>I see the above as not competing, but aspects of the same problem.</div><div><br></div><div><br></div></div>
<br></div></div><span class="">__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></span></blockquote></div><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>