[openstack-dev] [oslo][oslo.db] MySQL Cluster support

Doug Hellmann doug at doughellmann.com
Fri Feb 3 01:22:00 UTC 2017


Excerpts from Octave J. Orgeron's message of 2017-02-02 15:08:14 -0700:
> Comments below..
> 
> On 2/2/2017 1:08 PM, Doug Hellmann wrote:
> > Excerpts from Octave J. Orgeron's message of 2017-02-02 12:16:15 -0700:
> >> Hi Doug,
> >>
> >> Comments below..
> >>
> >> Thanks,
> >> Octave
> >>
> >> On 2/2/2017 11:27 AM, Doug Hellmann wrote:
> >>> Excerpts from Octave J. Orgeron's message of 2017-02-02 09:40:23 -0700:

[snip]

> > So all existing scripts that create or modify tables will need to
> > be updated? That's going to be a lot of work. It will also be a lot
> > of work to ensure that new alter scripts are implemented using the
> > required logic, and that testing happens in the gates for all
> > projects supporting this feature to ensure there are no regressions
> > or behavioral changes in the applications as a result of the changes
> > in table definitions.
> >
> > I'll let the folks more familiar with databases in general and MySQL
> > in particular respond to some of the technical details, but I think
> > I should give you fair warning that you're taking on a very big
> > project, especially for someone new to the community.
> 
> Yes, this is major undertaking and major driver for Oracle to setup a 
> 3rd party CI so that we can automate regression testing against MySQL 
> Cluster. On the flip side, it helps solve some of the challenges with 

I'm not sure we would want to gate projects on CI run outside of
our infrastructure community. We've had some bad experiences with
that in the past. What are the options for running MySQL Cluster
on nodes upstream?

> larger deployments where an active/passive solution for MySQL DB is not 
> sufficient. So the pay-off is pretty big from an availability and 
> scale-out perspective.
> 
> But I do realize that I'll have to maintain this long-term and hopefully 
> get others to help out as more services are added to OpenStack.

Given the scope of the work being proposed, and the level of expertise
needed to do it, we're going to need to have more than one person
available to debug issues before we go to far ahead with it.

It might help to have an example or two of the sorts of migration
script changes you've described elsewhere. Maybe you can prepare a
sample patch for a project?

Are there tools that can look at a table definition and tell if it
meets the criteria for the cluster backend (the row size, types
used, whatever else)? Or is it something one has to do by hand?

> > I may not have been entirely clear. You need to add a function to
> > oslo.db to allow a user of oslo.db to read the configuration value
> > without knowing what that option name is. There are two reasons for
> > this policy:
> >
> > 1. Configuration options are supposed to be completely transparent
> >     to the application developer using the library, otherwise they
> >     would be parameters to the classes or functions in the library
> >     instead of deployer-facing configuration options.
> >
> >     oslo.config allows us to rename configuration options transparently
> >     to deployers (they get a warning about the new name or location
> >     for the option in the config file, but the library knows both
> >     locations).
> >
> >     The rename feature does not work when accessing options
> >     programmatically, because we do not consider configuration options
> >     to be part of the API of a library.  That means that cfg.CONF.foo.bar
> >     can move to cfg.CONF.blah.bletch, and your code using it by the
> >     old name will break.
> 
> This is correct. Neutron does exactly what you are describing where you 
> have to look under a neutron namespace instead of the cfg.CONF namespace 
> to find the actual configured setting from the .conf file.

The neutron migration scripts and configuration options are "owned" by
the same code, so that's fine. That's not the case with oslo.db.

> > 2. Accessing configuration options depends on having them registered,
> >     and a user of the library that owns a configuration option may not
> >     know which functions in the library to call to register the options.
> >     As a result, they may try to use an option before it is actually
> >     defined. Using an access function to read the value of an option
> >     allows the library to ensure the option is registered before trying
> >     to return the value.
> >
> > For those reasons, in cases where a configuration option needs to
> > be exposed outside of the library we require a function defined
> > inside the library where we can have unit tests that will break if
> > the configuration option is renamed or otherwise changed, and so
> > we can handle those changes without breaking applications consuming
> > the library.
> >
> > In this case, the migration scripts are outside of oslo.db, so they
> > will need a public function added to oslo.db to access the configuration
> > value. The function should first ensure that the new option is
> > registered, and then return the configured value.
> 
> So what do you envision that I'll have to add to the oslo.db library to 
> make things work as you describe? What would be the best example for me 
> to build from?

You need a function that takes the cfg.CONF instance as an argument. It
should call register_opts() to register the option or options it uses,
and then it can return the value. Something like:

from oslo_db import options

def get_db_engine_type(conf):
    conf.register_opts(options.database_opts)
	return conf.database.mysql_storage_engine

Doug



More information about the OpenStack-dev mailing list