<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Drop.  It is wasting cycles, and not
      something we should use in production.  Migrations specific to
      SQLPlus are the most time consuming work-arounds we have.  SQLPlus
      does not suit our development approach.  <br>
      <br>
      <br>
      <br>
      On 02/03/2015 01:32 PM, Georgy Okrokvertskhov wrote:<br>
    </div>
    <blockquote
cite="mid:CAG_6_on9jf34xLV21KQtqmjzP85tkhW805m9Lech8+f_J7unMg@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div>
          <div>I think we should switch to clean migration path. We do
            have production installations but we can handle initial db
            uprgade case by case for customers. It is better to fix this
            issue now when we have few customers rather then doing later
            at larger scale.<br>
            <br>
          </div>
          Thanks<br>
        </div>
        Georgy<br>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Tue, Feb 3, 2015 at 9:05 AM, Mike
          Bayer <span dir="ltr"><<a moz-do-not-send="true"
              href="mailto:mbayer@redhat.com" target="_blank">mbayer@redhat.com</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex"><span
              class=""><br>
              <br>
              Andrew Pashkin <<a moz-do-not-send="true"
                href="mailto:apashkin@mirantis.com">apashkin@mirantis.com</a>>
              wrote:<br>
              <br>
              > Mike Bayer wrote:<br>
            </span><span class="">>> there’s always a naming
              convention in place; all databases other than<br>
              >> SQLite produce them on the fly if you don’t
              specify one.  The purpose<br>
              >> of the Alembic/SQLAlchemy naming_convention
              feature is so that you<br>
              >> have *one* naming convention, rather than N
              unpredictable conventions.<br>
              >> I’m not sure if you’re arguing the feature should
              not be used.  IMHO<br>
              >> it should definitely be used for an application
              that is deploying<br>
              >> cross-database.  Otherwise you have no choice but
              to hardcode the<br>
              >> naming conventions of each target database
              individually in all cases<br>
              >> that you need to refer to them.<br>
              > You can't just bring SA/Alembic "naming conventions"
              into the project,<br>
              > because they will collide with auto-generated
              constraint names.<br>
              <br>
            </span>I was proposing a way to fix this for the murano
            project which only appears to have four migrations so far,
            but with the assumption that there are existing production
            environments which cannot do a full rebuild.<br>
            <span class=""><br>
              ><br>
              > So you need to hardcode reverese-engineered
              constrants names into the<br>
              > old migrations and then add new migration that
              renames constraint<br>
              > according with "naming conventions”.<br>
              > OR you need to drop old<br>
              > migrations, and create new one with "naming
              conventions" - that will<br>
              > be backward incompatible, but cleaner.<br>
              <br>
              <br>
            </span>My proposal was to essentially do both strategies. 
             Build out fully clean migrations from the start, but also
            add an additional “conditional” migration that will repair a
            Postgresql / MySQL database that is already at the head, and
            is detected as having the older naming convention.  Because
            openstack does not appear to use offline migrations, this
            would be doable, though not necessarily worth it.<br>
            <br>
            If Murano can afford to just restart with clean migrations
            and has no production deployments yet which would be
            disrupted by a full rebuild, then sure, just do this.<br>
            <div class="HOEnZb">
              <div class="h5"><br>
                <br>
                <br>
                <br>
                <br>
                ><br>
                > On 03.02.2015 18:32, Mike Bayer wrote:<br>
                >> Andrew Pashkin <<a moz-do-not-send="true"
                  href="mailto:apashkin@mirantis.com">apashkin@mirantis.com</a>>
                wrote:<br>
                >><br>
                >>> Mike Bayer wrote:<br>
                >>>> The patch seems to hardcode the
                conventions for MySQL and Postgresql.<br>
                >>>> The first thought I had was that in
                order to remove the dependence<br>
                >>>> on them here, you’d need to instead
                simply turn off the<br>
                >>>> “naming_convention” in the MetaData if
                you detect that you’re on one<br>
                >>>> of those two databases. That would be a
                safer idea than trying to<br>
                >>>> hardcode these conventions (and would
                also work for other kinds<br>
                >>>> of backends).<br>
                >>> With your solution it is still will be
                necessary for developers<br>
                >>> to guess constraints names when writing new
                migrations. And it will<br>
                >>> be even harder, because they will need also
                to handle case of<br>
                >>> "naming conventions”.<br>
                >><br>
                >> there’s always a naming convention in place;
                all databases other than SQLite produce them on the fly
                if you don’t specify one.  The purpose of the
                Alembic/SQLAlchemy naming_convention feature is so that
                you have *one* naming convention, rather than N
                unpredictable conventions.   I’m not sure if you’re
                arguing the feature should not be used.  IMHO it should
                definitely be used for an application that is deploying
                cross-database.  Otherwise you have no choice but to
                hardcode the naming conventions of each target database
                individually in all cases that you need to refer to
                them.<br>
                >><br>
                >><br>
                >><br>
                >><br>
                >>> Mike Bayer wrote:<br>
                >>>> However, it’s probably worthwhile to
                introduce a migration that does<br>
                >>>> in fact rename existing constraints on
                MySQL and Postgresql.<br>
                >>> Yes, that's what I want to do in case of
                the first solution.<br>
                >>><br>
                >>> Mike Bayer wrote:<br>
                >>>>> Another possible solution is to
                drop all current migrations and<br>
                >>>>> introduce new one with correct
                names.<br>
                >>>> you definitely shouldn’t need to do
                that.<br>
                >>> Why?<br>
                >>><br>
                >>> On 30.01.2015 22:00, Mike Bayer wrote:<br>
                >>>> Andrew Pashkin <<a
                  moz-do-not-send="true"
                  href="mailto:apashkin@mirantis.com">apashkin@mirantis.com</a>>
                wrote:<br>
                >>>><br>
                >>>>> Working on this issue I encountered
                another problem.<br>
                >>>>><br>
                >>>>> Most indices in the project has no
                names and because of that,<br>
                >>>>> developer must reverse-engineer
                them in every migration.<br>
                >>>>> Read about that also here [1].<br>
                >>>>><br>
                >>>>> SQLAlchemy and Alembic provide
                feature for generation constraint<br>
                >>>>> names by pattern, specifically to
                resolve that kind of issues [1].<br>
                >>>>><br>
                >>>>> I decided to introduce usage of
                this feature in Murano.<br>
                >>>>><br>
                >>>>> I've implemented solution that
                preserves backward-compatibility<br>
                >>>>> for migration and allows to rename
                all constraints according<br>
                >>>>> to patterns safely [2]. With it
                user, that have already deployed Murano<br>
                >>>>> will be able to upgrade to new
                version of Murano without issues.<br>
                >>>>><br>
                >>>>> There are downsides in this
                solution:<br>
                >>>>> - It assumes that all versions of
                Postgres and MySQL uses the<br>
                >>>>> same patterns for constraints names
                generation.<br>
                >>>>> - It is hard to implement a test
                for this solution and it will be slow.<br>
                >>>>> Because there is need to reproduce
                such situation when user has old<br>
                >>>>> versions of migrations applied, and
                then tries to upgrade.<br>
                >>>><br>
                >>>> The patch seems to hardcode the
                conventions for MySQL and Postgresql.   The first
                thought I had was that in order to remove the dependence
                on them here, you’d need to instead simply turn off the
                “naming_convention” in the MetaData if you detect that
                you’re on one of those two databases.   That would be a
                safer idea than trying to hardcode these conventions
                (and would also work for other kinds of backends).<br>
                >>>><br>
                >>>> However, I’m not actually sure that you
                even need special behavior for these two backends.  If
                an operator runs these migrations on a clean database,
                then the constraints are generated with the consistent
                names on all backends.   if a target database already
                has these schema constructs present, then these
                migrations are never run; it doesn’t matter that they
                have the right or wrong names already.<br>
                >>>><br>
                >>>> I suppose then that the fear is that
                some PG/MySQL databases will have constraints that are
                named in one convention, and others will have
                constraints using the native conventions.    However,
                the case now is that all deployments are using native
                conventions, and being able to DROP these constraints is
                already not very feasible unless you again were willing
                to hardcode those naming conventions up forward.    The
                constraints in these initial migrations, assuming you
                don’t regenerate them, might just need to be left alone,
                and the project proceeds in the future with a consistent
                convention.<br>
                >>>><br>
                >>>> However, it’s probably worthwhile to
                introduce a migration that does in fact rename existing
                constraints on MySQL and Postgresql.  This would be a
                migration script that emits DROP CONSTRAINT and CREATE
                CONSTRAINT for all the above constraints that have an
                old name and a new name.  The script would need to check
                the backend, as you’re doing now, in order to run, and
                yes it would hardcode the names of those conventions,
                but at least it would just be a one-time run against
                only currently deployed databases.   Since your
                migrations are run “live”, the script can make itself a
                “conditional” run by checking for the “old” names and
                skipping those that don’t exist.<br>
                >>>><br>
                >>>>> Another possible solution is to
                drop all current migrations and<br>
                >>>>> introduce new one with correct
                names.<br>
                >>>><br>
                >>>> you definitely shouldn’t need to do
                that.<br>
                >>>><br>
                >>>><br>
                >>>>> This brings us to new problem -
                migrations and models are out of sync<br>
                >>>>> right now in multiple places -
                there are different field types in<br>
                >>>>> migrations and models, migrations
                introduces indices that is absent<br>
                >>>>> in models, etc.<br>
                >>>>><br>
                >>>>> And this solution has great
                downside - it is not backward-compatible,<br>
                >>>>> so all old users will lost their
                data.<br>
                >>>>><br>
                >>>>> We (Murano team) should decide,
                what solution we want to use.<br>
                >>>>><br>
                >>>>><br>
                >>>>> [1]<br>
                >>>>> <a moz-do-not-send="true"
href="http://alembic.readthedocs.org/en/latest/naming.html#tutorial-constraint-names"
                  target="_blank">http://alembic.readthedocs.org/en/latest/naming.html#tutorial-constraint-names</a><br>
                >>>>> [2] <a moz-do-not-send="true"
                  href="https://review.openstack.org/150818"
                  target="_blank">https://review.openstack.org/150818</a><br>
                >>>>><br>
                >>>>> --<br>
                >>>>> With kind regards, Andrew Pashkin.<br>
                >>>>> cell phone - <a
                  moz-do-not-send="true"
                  href="tel:%2B7%20%28985%29%20898%2057%2059"
                  value="+79858985759">+7 (985) 898 57 59</a><br>
                >>>>> Skype - waves_in_fluids<br>
                >>>>> e-mail - <a moz-do-not-send="true"
                  href="mailto:apashkin@mirantis.com">apashkin@mirantis.com</a><br>
                >>>>><br>
                >>>>>
__________________________________________________________________________<br>
                >>>>> OpenStack Development Mailing List
                (not for usage questions)<br>
                >>>>> Unsubscribe: <a
                  moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                  target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
                >>>>> <a moz-do-not-send="true"
                  href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                  target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
                >>>><br>
                >>>>
__________________________________________________________________________<br>
                >>>> OpenStack Development Mailing List (not
                for usage questions)<br>
                >>>> Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                  target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
                >>>> <a moz-do-not-send="true"
                  href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                  target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
                >>><br>
                >>> --<br>
                >>> With kind regards, Andrew Pashkin.<br>
                >>> cell phone - <a moz-do-not-send="true"
                  href="tel:%2B7%20%28985%29%20898%2057%2059"
                  value="+79858985759">+7 (985) 898 57 59</a><br>
                >>> Skype - waves_in_fluids<br>
                >>> e-mail - <a moz-do-not-send="true"
                  href="mailto:apashkin@mirantis.com">apashkin@mirantis.com</a><br>
                >>><br>
                >>>
__________________________________________________________________________<br>
                >>> OpenStack Development Mailing List (not for
                usage questions)<br>
                >>> Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                  target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
                >>> <a moz-do-not-send="true"
                  href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                  target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
                >><br>
                >>
__________________________________________________________________________<br>
                >> OpenStack Development Mailing List (not for
                usage questions)<br>
                >> Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                  target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
                >> <a moz-do-not-send="true"
                  href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                  target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
                ><br>
                > --<br>
                > With kind regards, Andrew Pashkin.<br>
                > cell phone - <a moz-do-not-send="true"
                  href="tel:%2B7%20%28985%29%20898%2057%2059"
                  value="+79858985759">+7 (985) 898 57 59</a><br>
                > Skype - waves_in_fluids<br>
                > e-mail - <a moz-do-not-send="true"
                  href="mailto:apashkin@mirantis.com">apashkin@mirantis.com</a><br>
                ><br>
                >
__________________________________________________________________________<br>
                > OpenStack Development Mailing List (not for usage
                questions)<br>
                > Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                  target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
                > <a moz-do-not-send="true"
                  href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                  target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
                <br>
__________________________________________________________________________<br>
                OpenStack Development Mailing List (not for usage
                questions)<br>
                Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                  target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
                <a moz-do-not-send="true"
                  href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                  target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
        <br clear="all">
        <br>
        -- <br>
        <div class="gmail_signature">
          <div dir="ltr"><font color="#999999"><span
                style="background-color:rgb(255,255,255)">Georgy
                Okrokvertskhov<br>
                Architect,<br>
                <span style="font-family:arial;font-size:small">OpenStack
                  Platform Products,</span><br>
                Mirantis</span><br>
              <a moz-do-not-send="true" href="http://www.mirantis.com/"
                target="_blank">http://www.mirantis.com</a><br>
              Tel. +1 650 963 9828<br>
              Mob. +1 650 996 3284</font><br>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>