Open Stack

Hello, Stackers.

Trove wiki and launchpad pages are stating that, it is a scalable  database
service that allows users to quickly and easily utilize the features of a
relational database without the burden of handling complex administrative
tasks. Trove already can provision single instances of certain databases.

How does developers can integrate new datastores?

Each datastore requires its own option group. What does it mean?

For each datastore developer has to defined
<https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L270-L436>
oslo
option group <https://wiki.openstack.org/wiki/Oslo/Config>. It contains a
set of required configuration parameters that are being used at various
stages of provisioning/management.

Group content

Each group contains options that are being used by regular Trove services
such as Trove-api, Trove-taskmanager and one very specific Trove-guestagent.

Options that are required by API and Taskmanager service:


   -

   tcp_ports - a list of TCP ports that would be used as basis for building
   rules for data security group assigned for instance.
   -

   upd_ports - a list of UDP ports that would be used as basis for building
   rules for data security group assigned for instance.
   -

   root_on_create - Enable the automatic creation of the root user for the
   service during instance-create. The generated password for the root user is
   immediately returned in the response of "instance-create as the 'password'
   field.
   -

   usage_timeout - Timeout to wait for a guest to become active.
   -

   backup_strategy - specific class that responsible for backup creation


    Options that are required by Trove-guestagent:


   -

   backup_strategy - specific class that responsible for backup creation
   -

   mount_point - FS path to mount block storage volume.
   -

   backup_namespace - backup namespace where backup_strategy defined
   -

   restore_namespace - restore namespace where restore strategy defined


For now all this options (for all services) are defined within one
configuration group.

So, here comes first suggestion. We need to split each datastore group onto
two:

   -

   server-side datastore options
   -

   guest-side datastore options


    Benefits. Refactoring would give an ability to split guest
per-datastore options and extract guest from codebase completely.

Now let’s take a look how does Trove-guestagent works.

Trove-guestagent is a RPC service with per-datastore manager. Each time
instance provisioning gets initiated server-side injects configuration
files, one of them contains significant option for guest -
datastore_manager option. It’s used to load specific datastore manager
implementation.

This is how it works:

    Type of configuration attribute: dictionary

    Name: datastore_registry_ext

    Example:

datastore_registry_ext = {percona:
trove.guestagent.datastore.mysql.manager.Manager,

                                     mysql:
trove.guestagent.datastore.mysql.manager.Manager,

                                    cassandra:
trove.guestagent.datastore.cassandra.manager.Manager,}

            Here comes an issue - each guest contains tons for files
specific to each datastore. For development reasons - it’s totally fine,
but for production requirements - it’s not good at all. Guest should be
lightweight, it should be small, etc.


How can we simplify datastore integration?

I’d like to propose to integrate stevedore
<http://stevedore.readthedocs.org/en/latest/> into Trove services.

>From Trove-guestagent perspective. According to description above we have
two separate entities that are needed to be injected during guest
deployment (while preparing image for specific datastore):

   -

   datastore configuration options
   -

   manager implementation


Basically, those entities can be merged into one, since manager
implementation relays on datastore configuration options.

>From Trove-API and Trove-taskmanager perspective we need to inject
per-datastore attributes which are mentioned above.

Implementation details

This topic mostly touches guestagent. Stevedore integration will forces us
to define new abstract layer for datastore manager - BaseDatastore manager.

Benefits? For now only mysql datastore manager can handle 100% of API calls
(see [1]) but other datastores would not support certain API calls (see [2]
- [5]). So, abstract layer would reduce size of certain manager
implementation (it would not contain calls like [6] ). Another one benefit
 - we don’t need to handle managers registry - each new datastore manager
can be included from 3d party packages through guests` setup.cfg

So, i’d like to initiate discussion with all of you folks before submitting
BP and requesting for review procedure.


Thoughts?

[1]
https://github.com/openstack/trove/blob/master/trove/guestagent/datastore/mysql/manager.py

[2]
https://github.com/openstack/trove/blob/master/trove/guestagent/datastore/mongodb/manager.py

[3]
https://github.com/openstack/trove/blob/master/trove/guestagent/datastore/couchbase/manager.py
<https://github.com/openstack/trove/blhttps://github.com/openstack/trove/blob/master/trove/guestagent/datastore/couchbase/manager.pyob/master/trove/guestagent/datastore/mysql/manager.py>

[4]
https://github.com/openstack/trove/blob/master/trove/guestagent/datastore/redis/manager.py

[5]
https://github.com/openstack/trove/blob/master/trove/guestagent/datastore/cassandra/manager.py

[6]
https://github.com/openstack/trove/blob/master/trove/guestagent/datastore/mongodb/manager.py#L96-L160



Best regards,

Denis Makogon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140724/facca88f/attachment.html>