<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <br>
    my initial proposal for scripted expand/contract migrations is up:<br>
    <br>
    <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/192937/">https://review.openstack.org/#/c/192937/</a><br>
    <br>
    <br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 6/18/15 5:54 AM, Anna Kamyshnikova
      wrote:<br>
    </div>
    <blockquote
cite="mid:CAOBv_tKfGf+qrg_ZLoaqguMsEdg204U5-nPXXXPTHeg42hvAUw@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Wed, Jun 17, 2015 at 8:23 PM, Mike
            Bayer <span dir="ltr"><<a moz-do-not-send="true"
                href="mailto:mbayer@redhat.com" target="_blank">mbayer@redhat.com</a>></span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000">
                <div>
                  <div class="h5"> <br>
                    <br>
                    <div>On 6/17/15 12:40 PM, Ihar Hrachyshka wrote:<br>
                    </div>
                    <blockquote type="cite">
                      <pre>-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 06/17/2015 11:27 AM, Anna Kamyshnikova wrote:
</pre>
                      <blockquote type="cite">
                        <pre>Ihar, thanks for bringing this up!

This is very interesting and I think it worth trying. I'm +1 on
that and want to participate in this work.

</pre>
                      </blockquote>
                      <pre>Awesome.

</pre>
                      <blockquote type="cite">
                        <pre>In fact a lot *not strict* migrations are removed with
juno_initial, so I hope it won't be so hard for now to apply
stricter rules for migration. But what is the plan for those
migrations that are *not strict* now?
</pre>
                      </blockquote>
                      <pre>Still, we have several live data migrations from Juno to Kilo:

- - 14be42f3d0a5_default_sec_group_table.py: populates db with default
security groups;

- - 16cdf118d31d_extra_dhcp_options_ipv6_support.py: populates
extraqdhcpopts with default ip_version = 4;

- - 2d2a8a565438_hierarchical_binding.py: populates db with
port_binding_levels objects, then drops old tables;

- - 35a0f3365720_add_port_security_in_ml2.py: port security field is
populated with True for ports and networks;

- - 034883111f_remove_subnetpool_allow_overlap.py: drops allow_overlap
column from subnetpools: probably unused so we can be ok with it?..

In any case, the plan for existing migration rules is: don't touch
them. Their presence in N release just indicates that we cannot get
online db migration in N+1. That's why we should adopt strict rules
the earlier the better, so that opportunity does not slip to N+X where
X is too far.

The patches currently in review that look suspicious in this regard are:
- - I4ff7db0f5fa12b0fd1c6b10318e9177fde0210d7: moves data from one table
into another;
- - Iecb3e168a805fc5f3d59d894c3f0d9298505e872: fills new columns with
default server values (why is it even needed?..);
- - Icde55742aa78ed995bac0896c01c80c9d28aa0cf: alter_column(). Not sure
we are ok with it;
- - I66b3ee8c2f9fa6f04b9e89dc49d1a3d277d63191: probably not an issue
though since it touches existing live data impact rule?</pre>
                    </blockquote>
                    <br>
                  </div>
                </div>
                <font color="#741b47"> I made a list of migrations from
                  juno-> kilo that are non expansive or do data
                  migrations:<br>
                  <br>
                </font>
                <div><span><b><font color="#741b47">these contain "drop
                        column":</font></b></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><font color="#741b47">034883111f_remove_subnetpool_allow_overlap.py</font></span></div>
                <div><span><font color="#741b47">2d2a8a565438_hierarchical_binding.py</font></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><b><font color="#741b47">these contain "drop
                        table":</font></b></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><font color="#741b47">28c0ffb8ebbd_remove_mlnx_plugin.py</font></span></div>
                <div><span><font color="#741b47">2b801560a332_remove_hypervneutronplugin_tables.py</font></span></div>
                <div><span><font color="#741b47">408cfbf6923c_remove_ryu_plugin.py</font></span></div>
                <div><span><font color="#741b47">57086602ca0a_scrap_nsx_adv_svcs_models.py</font></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><b><font color="#741b47">these contain data
                        migrations:</font></b></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><font color="#741b47">14be42f3d0a5_default_sec_group_table.py</font></span></div>
                <div><span><font color="#741b47">16cdf118d31d_extra_dhcp_options_ipv6_support.py</font></span></div>
                <div><span><font color="#741b47">2b801560a332_remove_hypervneutronplugin_tables.py</font></span></div>
                <div><span><font color="#741b47">2d2a8a565438_hierarchical_binding.py</font></span></div>
                <div><span><font color="#741b47">35a0f3365720_add_port_security_in_ml2.py</font></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><b><font color="#741b47">Example of failure:</font></b></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><font color="#741b47">neutron/db/migration/alembic_migrations/versions/2d2a8a565438_hierarchical_binding.py

                      - drops the following columns:</font></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><font color="#741b47">   
                      op.drop_constraint(fk_name_dvr[0],
                      'ml2_dvr_port_bindings', 'foreignkey')</font></span></div>
                <div><span><font color="#741b47">   
                      op.drop_column('ml2_dvr_port_bindings',
                      'cap_port_filter')</font></span></div>
                <div><span><font color="#741b47">   
                      op.drop_column('ml2_dvr_port_bindings', 'segment')</font></span></div>
                <div><span><font color="#741b47">   
                      op.drop_column('ml2_dvr_port_bindings', 'driver')</font></span></div>
                <div><font color="#741b47"><br>
                  </font></div>
                <div><span><font color="#741b47">   
                      op.drop_constraint(fk_name[0],
                      'ml2_port_bindings', 'foreignkey')</font></span></div>
                <div><span><font color="#741b47">   
                      op.drop_column('ml2_port_bindings', 'driver')</font></span></div>
                <div><span><font color="#741b47">   
                      op.drop_column('ml2_port_bindings', 'segment')</font></span></div>
                <div><span><font color="#741b47">    </font></span></div>
                <div><span><font color="#741b47">    which then causes a
                      failure in Juno:</font></span></div>
                <div><span><font color="#741b47">    </font></span></div>
                <div><span><font color="#741b47">    OperationalError:
                      (OperationalError) (1054, "Unknown column</font></span></div>
                <div><span><font color="#741b47">'ml2_port_bindings_1.driver'

                      in 'field list'") </font></span></div>
                <span class=""><font color="#741b47"> </font>
                  <div><font color="#741b47"><br>
                    </font></div>
                </span></div>
            </blockquote>
            <div> </div>
            <div><span style="color:rgb(0,0,0)">In M release we will be
                able to create kilo_initial migration that will hide all
                these migrations(juno -> kilo), so I as there is
                nothing we can do we won't touch them as Ihar said. The
                problem about migrations that are on review is more
                serious, as we should adopt strict rules as early as
                possible and all core reviewer should be aware about
                that.</span></div>
            <div><span style="color:rgb(0,0,0)"><br>
              </span></div>
            <div><span style="color:rgb(0,0,0)">To</span><span
                style="color:rgb(0,0,0)"> Ihar's list of migrations on
                review  I can add :</span></div>
            <div><font color="#000000">I3426b13eede8bfa29729cf3efea3419fb91175c4
                - insert some data, </font></div>
            <div><font color="#000000">I9cf36e1fd3a009c175e0d475af407a30f4e5c408
                (almost ready to be merged!) - drop tables.</font></div>
            <div><font color="#000000">And there are a lot changes with
                altering columns. One is merged in neutron-vpnaas.</font></div>
            <div><font color="#000000">So we should decide these things
                quickly. </font></div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000"><span class=""><br>
                  <br>
                  <br>
                  <blockquote type="cite">
                    <pre><font color="#741b47">(the list can be incomplete)

</font></pre>
                    <blockquote type="cite">
                      <pre><font color="#741b47">I think that we should try to use Alembic as much we could as Mike
is going to support us in that and we have time to make some change
in Alembic directly.
</font></pre>
                    </blockquote>
                    <pre><font color="#741b47">Yes, sure, I'm looking forward to see Mike's proposal in public.

</font></pre>
                    <blockquote type="cite">
                      <pre><font color="#741b47">We should undoubtedly plan this work for M release because there
will be some issues that will appear in the process.

</font></pre>
                    </blockquote>
                    <pre><font color="#741b47">Sure.

Ihar
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJVgaL4AAoJEC5aWaUY1u57oZgH/34pgV7AqOiq4XnWOOmQ9HA9
jL+8E9jv8pUSW3X4v0Rm5mDuWJyWscrgy61Om+sWsmqBFAmm/gmLWm+NNADbYM5e
6hsoaO5WmuvRc03MwIwsa0NEgfPc8EhT5JiZmYRjOBc85ZCs6+UOKUHBAI2EVTg8
t8YKdTdzxlrZQEOng1lbsUQYkHnNUZTbsREnpangfaHXBk3xmilH/ebGsz3CRUCe
OBrpp6q8N7mgZgK/UQKb04eS5bCna7eVmv6q7PvIO0SlYhhDbrL3+dv/SZpqQWZ/
Hek2Oig0IYyPygVrGc4BpT9MIaKisGoxXMn1rRB2g8us8jM58VyzqgXwEH2H4Aw=
=TqHb
-----END PGP SIGNATURE-----

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a moz-do-not-send="true" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</font></pre>
                  </blockquote>
                  <br>
                </span></div>
              <br>
__________________________________________________________________________<br>
              OpenStack Development Mailing List (not for usage
              questions)<br>
              Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
                rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
              <a moz-do-not-send="true"
                href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
                rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
              <br>
            </blockquote>
          </div>
          <br>
          <br clear="all">
          <div><br>
          </div>
          -- <br>
          <div class="gmail_signature">
            <div dir="ltr">Regards,
              <div>Ann Kamyshnikova</div>
              <div>Mirantis, Inc</div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>