Open Stack

On Thu, Sep 12, 2013 at 2:34 AM, Roman Podolyaka <rpodolyaka at mirantis.com>wrote:

> I can't agree more with Robert.
>
> Even if it was possible to downgrade all migrations without data loss, it
> would be required to make backups before DB schema upgrade/downgrade.
>
> E.g. MySQL doesn't support transactional DDL. So if a migration script
> can't be executed successfully for whatever reason (let's say we haven't
> tested it well enough on real data and it's turned out it has a few bugs),
> you will end up in a situation when the migration is partially applied...
> And migrations can possibly fail before backup tables are created, during
> this process or after it.
>

++ Data backups are a solved problem, and no DB admin should trust an
application to perform its own backups. If the application-specific
migration code is buggy (and therefore you need to restore from a
backup...), would you really trust the application-specific backup solution
to be any more reliable?


>
> Thanks,
> Roman
>
>
> On Thu, Sep 12, 2013 at 8:30 AM, Robert Collins <robertc at robertcollins.net
> > wrote:
>
>> I think having backup tables adds substantial systematic complexity,
>> for a small use case.
>>
>> Perhaps a better answer is to document in 'take a backup here' as part
>> of the upgrade documentation and let sysadmins make a risk assessment.
>> We can note that downgrades are not possible.
>>
>> Even in a public cloud doing trunk deploys, taking a backup shouldn't
>> be a big deal: *those* situations are where you expect backups to be
>> well understood; and small clouds don't have data scale issues to
>> worry about.
>>
>> -Rob
>>
>> -Rob
>>
>> On 12 September 2013 17:09, Joshua Hesketh <joshua.hesketh at rackspace.com>
>> wrote:
>> > On 9/4/13 6:47 AM, Michael Still wrote:
>> >>
>> >> On Wed, Sep 4, 2013 at 1:54 AM, Vishvananda Ishaya
>> >> <vishvananda at gmail.com> wrote:
>> >>>
>> >>> +1 I think we should be reconstructing data where we can, but keeping
>> >>> track of
>> >>> deleted data in a backup table so that we can restore it on a
>> downgrade
>> >>> seems
>> >>> like overkill.
>> >>
>> >> I guess it comes down to use case... Do we honestly expect admins to
>> >> regret and upgrade and downgrade instead of just restoring from
>> >> backup? If so, then we need to have backup tables for the cases where
>> >> we can't reconstruct the data (i.e. it was provided by users and
>> >> therefore not something we can calculate).
>> >
>> >
>> > So assuming we don't keep the data in some kind of backup state is
>> there a
>> > way we should be documenting which migrations are backwards
>> incompatible?
>> > Perhaps there should be different classifications for data-backwards
>> > incompatible and schema incompatibilities.
>> >
>> > Having given it some more thought, I think I would like to see
>> migrations
>> > keep backups of obsolete data. I don't think it is unforeseeable that an
>> > administrator would upgrade a test instance (or less likely, a
>> production)
>> > by accident or not realising their backups are corrupted, outdated or
>> > invalid. Being able to roll back from this point could be quite useful.
>> I
>> > think potentially more useful than that though is that if somebody ever
>> > needs to go back and look at some data that would otherwise be lost it
>> is
>> > still in the backup table.
>> >
>> > As such I think it might be good to see all migrations be downgradable
>> > through the use of backup tables where necessary. To couple this I
>> think it
>> > would be good to have a standard for backup table naming and maybe
>> schema
>> > (similar to shadow tables) as well as an official list of backup tables
>> in
>> > the documentation stating which migration they were introduced and how
>> to
>> > expire them.
>> >
>> > In regards to the backup schema, it could be exactly the same as the
>> table
>> > being backed up (my preference) or the backup schema could contain just
>> the
>> > lost columns/changes.
>> >
>> > In regards to the name, I quite like "backup_table-name_migration_214".
>> The
>> > backup table name could also contain a description of what is backed up
>> (for
>> > example, 'uuid_column').
>> >
>> > In terms of expiry they could be dropped after a certain
>> release/version or
>> > left to the administrator to clear out similar to shadow tables.
>> >
>> > Thoughts?
>> >
>> > Cheers,
>> > Josh
>> >
>> > --
>> > Rackspace Australia
>> >
>> >>
>> >> Michael
>> >>
>> >
>> >
>> > _______________________________________________
>> > OpenStack-dev mailing list
>> > OpenStack-dev at lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> Robert Collins <rbtcollins at hp.com>
>> Distinguished Technologist
>> HP Converged Cloud
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

-Dolph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130912/895096ea/attachment.html>