<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Doug,<br>
<br>
Comments below..<br>
<br>
Thanks,<br>
Octave<br>
<br>
<div class="moz-cite-prefix">On 2/2/2017 11:27 AM, Doug Hellmann
wrote:<br>
</div>
<blockquote cite="mid:1486059266-sup-3402@lrrr.local" type="cite">
<pre wrap="">Excerpts from Octave J. Orgeron's message of 2017-02-02 09:40:23 -0700:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Doug,
One could try to detect the default engine. However, in MySQL Cluster,
you can support multiple storage engines. Only NDB is fully clustered
and replicated, so if you accidentally set a table to be InnoDB it won't
be replicated . So it makes more sense for the operator to be explicit
on which engine they want to use.
</pre>
</blockquote>
<pre wrap="">
I think this change is probably a bigger scale item than I understood
it to be when you originally contacted me off-list for advice about
how to get started. I hope I haven't steered you too far wrong, but
at least the conversation is started.
As someone (Mike?) pointed out on the review, the option by itself
doesn't do much of anything, now. Before we add it, I think we'll
want to see some more detail about how it's going used. It may be
easier to have that broader conversation here on email than on the
patch currently up for review.</pre>
</blockquote>
<br>
Understood, it's a complicated topic since it involves gritty
details in SQL Alchemy and Alembic that are masked from end-users
and operators alike. Figuring out how to make this work did take
some time on my part.<br>
<br>
<blockquote cite="mid:1486059266-sup-3402@lrrr.local" type="cite">
<pre wrap="">
It sounds like part of the plan is to use the configuration setting
to control how the migration scripts create tables. How will that
work? Does each migration need custom logic, or can we build helpers
into oslo.db somehow? Or will the option be passed to the database
to change its behavior transparently?</pre>
</blockquote>
<br>
These are good questions. For each service, when the db sync or db
manage operation is done it will call into SQL Alchemy or Alembic
depending on the methods used by the given service. For example,
most use SQL Alchemy, but there are services like Ironic and Neutron
that use Alembic. It is within these scripts under the
<service>/db/* hierarchy that the logic exist today to
configure the database schema for any given service. Both approaches
will look at the schema version in the database to determine where
to start the create, upgrade, heal, etc. operations. What my patches
do is that in the scripts where a table needs to be modified, there
will be custom IF/THEN logic to check the
cfg.CONF.database.mysql_storage_engine setting to make the required
modifications. There are also use cases where the api.py or
model(s).py under the <service>/db/ hierarchy needs to look at
this setting as well for API and CLI operations where mysql_engine
is auto-inserted into DB operations. In those use cases, I replace
the hard coded "InnoDB" with the mysql_storage_engine variable.<br>
<br>
It would be interesting if we could develop some helpers to automate
this, but it would probably have to be at the SQL Alchemy or Alembic
levels. Unfortunately, throughout all of the OpenStack services
today we are hard coding things like mysql_engine, using InnoDB
specific features (savepoints, nested operations, etc.), and not
following the strict SQL orders for modifying table elements
(foreign keys, constraints, and indexes). That actually makes it
difficult to support other MySQL dialects or other databases out of
the box. SQL Alchemy can be used to fix some of these things if the
SQL statements are all generic and we follow strict SQL rules. But
to change that would be a monumental effort. That is why I took this
approach of just adding custom logic. There is a president for this
already for Postgres and DB2 support in some of the OpenStack
services using custom logic to deal with similar differences.<br>
<br>
As to why we should place the configuration setting into oslo.db?
Here are a couple of logical reasons:<br>
<ul>
<li>The configuration block for database settings for each service
comes from the oslo.db namespace today under
cfg.CONF.database.*. Placing it here makes the location
consistent across all of the services.<br>
</li>
<li>Within the SQL Alchemy and Alembic scripts, this is one of the
few common namespaces that are available without bringing in a
larger number of modules across the services today. <br>
</li>
<li>Many of the SQL Alchemy and Alembic scripts only import the
minimal set of python modules. If we imported others, we would
also have to initialize those name spaces which means a lot more
code :(<br>
</li>
<li>Reduces the amount of overhead required to make these changes.<br>
</li>
</ul>
<br>
<blockquote cite="mid:1486059266-sup-3402@lrrr.local" type="cite">
<pre wrap="">
Keep in mind that we do not encourage code outside of libraries to
rely on configuration settings defined within libraries, because
that limits our ability to change the names and locations of the
configuration variables. If migration scripts need to access the
configuration setting we will need to add some sort of public API
to oslo.db to query the value. The function can simply return the
configured value.</pre>
</blockquote>
<br>
Configuration parameters within any given service will make use of a
large namespace that pulls in things from oslo and the .conf files
for a given service. So even when an API, CLI, or DB related call is
made, these namespaces are key for things to work. In the case of
the SQL Alchemy and Alembic scripts, they also make use of this
namespace with oslo, oslo.db, etc. to figure out how to connect to
the database and other database settings. I don't think we need a
public API for these kinds of calls as the community already makes
use of the libraries to build the namespace. My oslo.db setting and
patches for each service just make use of the cfg.CONF.database
namespace to determine the correct behavior to execute.<br>
<br>
<blockquote cite="mid:1486059266-sup-3402@lrrr.local" type="cite">
<pre wrap="">
What other behaviors are likely to be changed by the new option?
Will application runtime behavior need to know about the storage
engine?</pre>
</blockquote>
<br>
The changes will be transparent to the application runtime behavior.
The APIs and CLI tools call into the <service>/db/api.py as
the entry point for database calls. Behind this you usually have a
models.py that is aware of the database schema to understand the
layout of things. So the underlining structure is abstracted away
from the run-time. These entry points sometimes do require minor
modifications to handle any hard coded issues or intercept functions
like savepoints and nested operations. Again I use the
cfg.CONF.database namespace to check for the appropriate behavior
and implement IF/THEN logic to do the right thing. <br>
<br>
Some of my design objectives for all of these patches are:<br>
<ul>
<li>Zero impact on OpenStack functionality and usability (API,
CLI, user experience)</li>
<li>No loss in database structure. Consistent foreign keys,
constraints, indexes, etc.</li>
<li>Minimal impact on column size and/or types to fit within NDB
table row limits. Many columns are over-sized today.</li>
<li>Validate functionality of APIs, service processes, and CLI.
Tempest is our friend :)<br>
</li>
<li>Zero impact for users not using MySQL Cluster (NDB).<br>
</li>
</ul>
<br>
<blockquote cite="mid:1486059266-sup-3402@lrrr.local" type="cite">
<pre wrap="">
Doug
</pre>
<blockquote type="cite">
<pre wrap="">
Thanks,
Octave
On 2/2/2017 6:46 AM, Doug Hellmann wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Excerpts from Octave J. Orgeron's message of 2017-02-01 20:33:38 -0700:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Folks,
I'm working on adding support for MySQL Cluster to the core OpenStack
services. This will enable the community to benefit from an
active/active, auto-sharding, and scale-out MySQL database. My approach
is to have a single configuration setting in each core OpenStack service
in the oslo.db configuration section called mysql_storage_engine that
will enable the logic in the SQL Alchemy or Alembic upgrade scripts to
handle the differences between InnoDB and NDB storage engines
respectively. When enabled, this logic will make the required table
schema changes around:
* Row character length limits 65k -> 14k
* Proper SQL ordering of foreign key, constraints, and index operations
* Interception of savepoint and nested operations
By default this functionality will not be enabled and will have no
impact on the default InnoDB functionality. These changes have been
tested on Kilo and Mitaka in previous releases of our OpenStack
distributions with Tempest. I'm working on updating these patches for
upstream consumption. We are also working on a 3rd party CI for
regression testing against MySQL Cluster for the community.
The first change set is for oslo.db and can be reviewed at:
<a class="moz-txt-link-freetext" href="https://review.openstack.org/427970">https://review.openstack.org/427970</a>
Thanks,
Octave
</pre>
</blockquote>
<pre wrap="">Is it possible to detect the storage engine at runtime, instead of
having the operator configure it?
Doug
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<br>
<p>
<a href="http://www.oracle.com/" target="_blank"><img
src="cid:part1.02050204.08030804@oracle.com" alt="Oracle"
border="0" height="26" width="114"></a><br>
<font color="#666666" size="2" face="Verdana, Arial, Helvetica,
sans-serif">Octave J. Orgeron | Sr. Principal Architect and
Software Engineer<br>
<font color="#ff0000">Oracle</font> Linux OpenStack<br>
Mobile: <a href="tel:+17206161550">+1-720-616-1550</a><br>
500 Eldorado Blvd. | Broomfield, CO 80021<br>
<a
href="http://www.oracle.com/us/solutions/enterprise-architecture/index.html"><img
src="cid:part4.07030303.06050903@oracle.com"
alt="Certified Oracle Enterprise Architect: Systems
Infrastructure" border="0" height="42" width="182"></a><br>
<a href="http://www.oracle.com/commitment" target="_blank"><img
src="cid:part6.04050004.07000905@oracle.com" alt="Green
Oracle" align="absmiddle" border="0" height="28"
width="44"></a>
<font color="#4B7D42" size="1" face="Verdana, Arial,
Helvetica, sans-serif">Oracle is committed to developing
practices and products that help protect the environment</font>
</font></p>
<font color="#666666" size="2" face="Verdana, Arial, Helvetica,
sans-serif">
</font></div>
</body>
</html>