Open Stack

Design: What is a capability?
A capabilities is an attribute of a datastore/version that allows dynamic code flow for different datastore/versions, The presence of a capability determines which code path is taken. A capability is an ID, a friendly name (used in our code), and a lengthy description (for displaying to the user).

Design: Where do they live?
Capabilities will live in their own table. In addition, there will be an association table to map between datastore versions and capabilities.

Design: Why isn't this in config files?
First, they will be able to be viewed by customers, and modified by operations. That does not lend itself to config files. Additionally, we should be able to change a datastores capabilities without a restart.

Operations: What happens if the capabilities table is not populated?
We are setting everything to be default off for all features. For mysql to support ephemeral_volumes it would need to be added to the capabilities table and properly associated to each version of mysql.

Operations: How do you manage them?
For the first pass, trove-manage commands. There will be a add_capability, remove_capability, associate_capability_with_datastore, and unassociate_capability_from_datastore (name are completely off the top of my head and subject to discussion/change).

Development: When/where are capabilities checked/determined?
Any time you would want to fork paths. The check for ephemeral_volume would happen right before the cinder volume is created, whereas the database capability would be checked as soon as possible in the api layer.

Development: How are capabilities checked?
This is still to be determined.

Option one (the one discussed at the meetup):
    if(cfg.cababilites.mysql.ephemeral_volumes):
        do_code()

Option two:
    if "ephemeral_volumes" in datastore.manager.capabilities)
        do_code()

The advantages of option one is consistency in checking the cfg object for configuration like things. I think that is worse than checking a property of the datastore. The advantage of the second way is that is more cleanly expresses what is happening under the hood without any additional cost. I have the second option about complete, so we can see what that looks like soon.

Most of this was discussed and agreed upon last week, I just wanted to recap and confirm with everyone on what our objectives and goals are. If there are unanswered questions, lets discuss sooner rather than later.


- KPom


Bonus: API discussion for future development (not right now)
I suggest we expose only the capabilities that belong to a specific datastore. As a customer, I shouldn't know every feature available, just the ones on the datastore they are looking at.

GET /{tenant_id}/datastores/{datastore_version_id}/capabilities


Eventually, we would want the following admin routes available, but that is mitigated by the trove-manage stuff.

[GET | PUT | POST | DELETE] /capabilities
[POST | DELETE] /datastores/datastore_version_id/capabilities





________________________________
From: Denis Makogon [dmakogon at mirantis.com]
Sent: Wednesday, February 19, 2014 1:00 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [TROVE] Trove capabilities

I know that Kaleb is working on that.

But we(me, Kaleb, rest of guys) decided that i'm gonna present my own vision of Capabilities.
That is what i'm doing now. Basicaly, at meetup you all could discuss design in general, i suppose.

Best regards,
Denis Makogon.


On Wed, Feb 19, 2014 at 7:42 PM, Daniel Salinas <imsplitbit at gmail.com<mailto:imsplitbit at gmail.com>> wrote:
We're at the trove meetup for the rest of this week but you should speak with Kaleb Pomeroy from Rackspace.  He's generated some code on this particular subject.


On Wed, Feb 19, 2014 at 11:11 AM, Denis Makogon <dmakogon at mirantis.com<mailto:dmakogon at mirantis.com>> wrote:

Goodday OpenStack DBaaS communtiy.


I’d like to start topic related to design of capabilities [1].


This question was raised when multiple datastores where integrated into Trove (such as redis, cassandra, mongo in instance modes). We encountered a the problem when specific datastore doesn’t support operations that are the part of Trove API (core and extension). After some discussions with the community we decided to implement “capabilities” feature. Basicaly, “capabilities” is the list of operations acceptable for chosen “object”. To be accurate,  “capabilities” is the API for listing available Trove API(core and extensions) per some kind of an identifier.

Design.


I’d like to split up all tasks into small peaces:


  1.  Nameing convintion.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.dqx5s9nxd86g>

  2.  DSL for writing reference to available capabilities.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.a86gyngvxgek>

  3.  Сustody strategy. How and what to store at the back-end?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.p3kja3s8zex>

  4.  Pinning to datastore or version?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.mvusz3lj5res>

  5.  How to load  actual API reference?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.df6yrbqny8te>

  6.  Default capabilities. What operations are allowed by default?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.gjahbyoqoq5d>

  7.  Capablities API.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.ihzh5pv7pmci>

  8.  Capabilities Management API.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.paouley6yndv>

Naming convention


Suggestions:

  1.  Filename convension:

{datastore or datastore_version_manager}.capabilities

  1.  Attribute convension:

Trove-”API-section”:

- method: Identifier

    Available sections: instance, backip, users, schemes.

    Capability Identifier: ALLOWED/BLOCKED

DSL for writing reference to available capabilities


There are several options of DSL’s that can be chosen for writing capabilities references, options:

  1.  JSON.

  2.  YAML.

  3.  XML.

    Personaly i’d like to choose YAML as appropriate DSL. Described capabilities would look like: mysql.capabilities

Trove-Instance:

- create: ALLOWED

- list: BLOCKED

- delete: ALLOWED

Trove-Backup:

- create: BLOCKED

Trove-Users:

- create: BLOCKED

Сustody strategy. How and what to store at the back-end ?


Before suggesting database table scheme i’d like to describe what we are going to store. It would be better to store only blocked capabilities, its cheaper, instead of storing all capabilities, either blocked and allowed.

Backend table scheme.


Table 1 - Capabilities table scheme

Datastore or Datastore version manager


Capabilities


datastore or manager



“Trove-”API-section”:

                     - method: Identifier“



    Where Identifier could be ALLOWED or BLOCKED

Pinning to datastore or datastore version?


As you can see from previous topics, it would be better to pin capabilities directly to datastore version, since one manager could be assigned to multiple datastore versions.

    So, capabilities backend table scheme would have next desription:


  *   datastrove_version_manager:

name: manager, Foreing key from DatastoreVersion table;

type: String.

  *   capabilities:

name: capabilities;

type: Text.

How to load actual API reference?


    According to previous topics i forsee the next way of discovering actual capabilities at runtime:

    Since YAML format perfectly could be loaded as Python dictionary, the easiest way is to merge two dictionaries.

    Example:


    default_capabilities = (CapabilitiesModel.

            load_defaults(manager=datastore_version.manager))

blocked_capabilities = (CapabilitiesModel.

load_blocked(manager=datastore_version.manager))

    actual_capabilities = dict(chain(

default_capabilities.iteritems(),

blocked_capabilities.iteritems()

)

)



Default capabilities. What operations are allowed by default?



Lets take a look at those datastore types that Trove supports. For now it supports: mysql, redis, cassandra, mongo.

    Not so long ago community team decided to bring up Capablities matrix [2]. By default each datastore should support core API [3] and extensions API [4] are optional. So, i’d like to suggest to mark core API capabilities as ALLOWED, and extensions API capabilities as BLOCKED.

Capabilities API


Task:     describe actual capabilities per datastore version manager

HTTP method:     GET

Method name:     show

Route:     /{tenant_id}/capabilities/{datastore_version_id}

CLI call:     trove capabilities-show --datastore-version <UUID>


Task:     list all actual capabilities

HTTP method:     GET

Method name:     index

Route:     /{tenant_id}/capabilities/

CLI call:     trove capabilities-list

Capabilities Management API



My suggestion is to add capabilities management API to trove-manage utility (for the first iteration).


trove-manage capabilities_update <datastore_manager> <Trove-”API-section”> <method> <Identifier>

where:

<datastore_manager> - datastore manager (variants: mysql, redis, cassandra, mongo, percona);

<Trove-”API-section”>  - part of API (core or extension, possible variants: Trove-Instance, Trove-Backup, Trove-Users, etc);

<method> - method available for given section of Trove API (variants: create, delete, list, show etc)

    <Identifier> - identifier that marks method from API section as ALLOWED/BLOCKED



[1] https://wiki.openstack.org/wiki/Trove/trove-capabilities

[2] https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix

[3] https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix#API_Matrix

[4] https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix#Extensions_Matrix



Best regards

Denis Makogon.
<https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix#Extensions_Matrix>


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140225/884d85a6/attachment-0001.html>