[openstack-dev] [TROVE] Trove capabilities
Denis Makogon
dmakogon at mirantis.com
Wed Feb 26 11:18:27 UTC 2014
First of all, how does Trove knows what kind of capabilities it can give to
users ?
We should have some kind of identity, that stores the restricted
description of all capabilities. Suggested variant is not acceptable,
because you could register any kind of capabilities in database.
Main point: Trove should have restrict description of all possible
capabilities, and it’s should not be stored in backend table.
Design: What is a capability? - Almost totally disagree.
Only datastore vesrion association is significant, common capabilities
table is not needed at all, because all new features(capabilities) assums
that Trove will be re-deployed from scratch. It means that capabilities
description would be updated only 2 or 3 time per release.
Design: Why isn't this in config files? - Agreed, my solution design
doesn’t required restart. Magic happes in runtime.
Development: How are capabilities checked? - Option 2 is more preferable,
because capabilities should be stored as dictionary object that has next
structure:
API call: ALLOWED/BLOCKED
Unfortunately given design is too broad, i dont see any implementation
steps description. Please provide more details and flow examples with
pseudo-code.
Best regards,
Denis Makogon
On Wed, Feb 26, 2014 at 12:13 AM, Kaleb Pomeroy <kaleb.pomeroy at rackspace.com
> wrote:
>
> *Design: What is a capability?*
> A capabilities is an attribute of a datastore/version that allows dynamic
> code flow for different datastore/versions, The presence of a capability
> determines which code path is taken. A capability is an ID, a friendly name
> (used in our code), and a lengthy description (for displaying to the user).
>
> *Design: Where do they live?*
> Capabilities will live in their own table. In addition, there will be an
> association table to map between datastore versions and capabilities.
>
>
> *Design: Why isn't this in config files? *First, they will be able to be
> viewed by customers, and modified by operations. That does not lend itself
> to config files. Additionally, we should be able to change a datastores
> capabilities without a restart.
>
> *Operations: What happens if the capabilities table is not populated?*
> We are setting everything to be *default off* for all features. For mysql
> to support ephemeral_volumes it would need to be added to the capabilities
> table and properly associated to each version of mysql.
>
> *Operations: How do you manage them?*
> For the first pass, trove-manage commands. There will be a add_capability,
> remove_capability, associate_capability_with_datastore, and
> unassociate_capability_from_datastore (name are completely off the top of
> my head and subject to discussion/change).
>
> *Development: When/where are capabilities checked/determined?*
> Any time you would want to fork paths. The check for ephemeral_volume
> would happen right before the cinder volume is created, whereas the
> database capability would be checked as soon as possible in the api layer.
>
> *Development: How are capabilities checked?*
> This is still to be determined.
>
> Option one (the one discussed at the meetup):
> if(cfg.cababilites.mysql.ephemeral_volumes):
> do_code()
>
> Option two:
> if "ephemeral_volumes" in datastore.manager.capabilities)
> do_code()
>
> The advantages of option one is consistency in checking the cfg object for
> configuration like things. I think that is worse than checking a property
> of the datastore. The advantage of the second way is that is more cleanly
> expresses what is happening under the hood without any additional cost. I
> have the second option about complete, so we can see what that looks like
> soon.
>
> Most of this was discussed and agreed upon last week, I just wanted to
> recap and confirm with everyone on what our objectives and goals are. If
> there are unanswered questions, lets discuss sooner rather than later.
>
>
> - KPom
>
>
>
> * Bonus: API discussion for future development (not right now) *I suggest
> we expose only the capabilities that belong to a specific datastore. As a
> customer, I shouldn't know every feature available, just the ones on the
> datastore they are looking at.
>
> GET /{tenant_id}/datastores/{datastore_version_id}/capabilities
>
>
> Eventually, we would want the following admin routes available, but that
> is mitigated by the trove-manage stuff.
>
> [GET | PUT | POST | DELETE] /capabilities
> [POST | DELETE] /datastores/datastore_version_id/capabilities
>
>
>
>
>
> ------------------------------
> *From:* Denis Makogon [dmakogon at mirantis.com]
> *Sent:* Wednesday, February 19, 2014 1:00 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [TROVE] Trove capabilities
>
> I know that Kaleb is working on that.
>
> But we(me, Kaleb, rest of guys) decided that i'm gonna present my own
> vision of Capabilities.
> That is what i'm doing now. Basicaly, at meetup you all could discuss
> design in general, i suppose.
>
> Best regards,
> Denis Makogon.
>
>
> On Wed, Feb 19, 2014 at 7:42 PM, Daniel Salinas <imsplitbit at gmail.com>wrote:
>
>> We're at the trove meetup for the rest of this week but you should speak
>> with Kaleb Pomeroy from Rackspace. He's generated some code on this
>> particular subject.
>>
>>
>> On Wed, Feb 19, 2014 at 11:11 AM, Denis Makogon <dmakogon at mirantis.com>wrote:
>>
>>> Goodday OpenStack DBaaS communtiy.
>>>
>>> I’d like to start topic related to design of capabilities [1].
>>>
>>> This question was raised when multiple datastores where integrated
>>> into Trove (such as redis, cassandra, mongo in instance modes). We
>>> encountered a the problem when specific datastore doesn’t support
>>> operations that are the part of Trove API (core and extension). After some
>>> discussions with the community we decided to implement “capabilities”
>>> feature. Basicaly, “capabilities” is the list of operations acceptable for
>>> chosen “object”. To be accurate, “capabilities” is the API for listing
>>> available Trove API(core and extensions) per some kind of an identifier.
>>>
>>> Design.
>>>
>>> I’d like to split up all tasks into small peaces:
>>>
>>>
>>> 1.
>>>
>>> Nameing convintion.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.dqx5s9nxd86g>
>>> 2.
>>>
>>> DSL for writing reference to available capabilities.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.a86gyngvxgek>
>>> 3.
>>>
>>> Сustody strategy. How and what to store at the back-end?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.p3kja3s8zex>
>>> 4.
>>>
>>> Pinning to datastore or version?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.mvusz3lj5res>
>>> 5.
>>>
>>> How to load actual API reference?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.df6yrbqny8te>
>>> 6.
>>>
>>> Default capabilities. What operations are allowed by default?<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.gjahbyoqoq5d>
>>> 7.
>>>
>>> Capablities API.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.ihzh5pv7pmci>
>>> 8.
>>>
>>> Capabilities Management API.<https://docs.google.com/a/mirantis.com/document/d/1FDhenfF47UYdiqTr5dGHSG_Gk828yFuvew92I9qil5I/edit#heading=h.paouley6yndv>
>>>
>>>
>>> Naming convention
>>>
>>> Suggestions:
>>>
>>> 1.
>>>
>>> Filename convension:
>>>
>>> {datastore or datastore_version_manager}.capabilities
>>>
>>> 1.
>>>
>>> Attribute convension:
>>>
>>> Trove-”API-section”:
>>>
>>> - method: Identifier
>>>
>>> Available sections: instance, backip, users, schemes.
>>>
>>> Capability Identifier: ALLOWED/BLOCKED
>>> DSL for writing reference to available capabilities
>>>
>>> There are several options of DSL’s that can be chosen for writing
>>> capabilities references, options:
>>>
>>> 1.
>>>
>>> JSON.
>>> 2.
>>>
>>> YAML.
>>> 3.
>>>
>>> XML.
>>>
>>> Personaly i’d like to choose YAML as appropriate DSL. Described
>>> capabilities would look like: mysql.capabilities
>>>
>>> Trove-Instance:
>>>
>>> - create: ALLOWED
>>>
>>> - list: BLOCKED
>>>
>>> - delete: ALLOWED
>>>
>>> Trove-Backup:
>>>
>>> - create: BLOCKED
>>>
>>> Trove-Users:
>>>
>>> - create: BLOCKED
>>>
>>> Сustody strategy. How and what to store at the back-end ?
>>>
>>> Before suggesting database table scheme i’d like to describe what we are
>>> going to store. It would be better to store only blocked capabilities, its
>>> cheaper, instead of storing all capabilities, either blocked and allowed.
>>>
>>> Backend table scheme.
>>>
>>> Table 1 - Capabilities table scheme
>>>
>>> Datastore or Datastore version manager
>>>
>>> Capabilities
>>>
>>> datastore or manager
>>>
>>> “Trove-”API-section”:
>>>
>>> - method: Identifier“
>>>
>>> Where Identifier could be ALLOWED or BLOCKED
>>> Pinning to datastore or datastore version?
>>>
>>> As you can see from previous topics, it would be better to pin
>>> capabilities directly to datastore version, since one manager could be
>>> assigned to multiple datastore versions.
>>>
>>> So, capabilities backend table scheme would have next desription:
>>>
>>>
>>> -
>>>
>>> datastrove_version_manager:
>>>
>>> name: manager, Foreing key from DatastoreVersion table;
>>>
>>> type: String.
>>>
>>> -
>>>
>>> capabilities:
>>>
>>> name: capabilities;
>>>
>>> type: Text.
>>>
>>> How to load actual API reference?
>>>
>>> According to previous topics i forsee the next way of discovering
>>> actual capabilities at runtime:
>>>
>>> Since YAML format perfectly could be loaded as Python dictionary,
>>> the easiest way is to merge two dictionaries.
>>>
>>> Example:
>>>
>>> default_capabilities = (CapabilitiesModel.
>>>
>>> load_defaults(manager=datastore_version.manager))
>>>
>>> blocked_capabilities = (CapabilitiesModel.
>>>
>>> load_blocked(manager=datastore_version.manager))
>>>
>>> actual_capabilities = dict(chain(
>>>
>>> default_capabilities.iteritems(),
>>>
>>> blocked_capabilities.iteritems()
>>>
>>> )
>>>
>>> )
>>>
>>>
>>>
>>> Default capabilities. What operations are allowed by default?
>>>
>>>
>>>
>>> Lets take a look at those datastore types that Trove supports. For now
>>> it supports: mysql, redis, cassandra, mongo.
>>>
>>> Not so long ago community team decided to bring up Capablities
>>> matrix [2]. By default each datastore should support core API [3] and
>>> extensions API [4] are optional. So, i’d like to suggest to mark core API
>>> capabilities as ALLOWED, and extensions API capabilities as BLOCKED.
>>> Capabilities API
>>>
>>> Task: describe actual capabilities per datastore version manager
>>>
>>> HTTP method: GET
>>>
>>> Method name: show
>>>
>>> Route: /{tenant_id}/capabilities/{datastore_version_id}
>>>
>>> CLI call: trove capabilities-show --datastore-version <UUID>
>>>
>>> Task: list all actual capabilities
>>>
>>> HTTP method: GET
>>>
>>> Method name: index
>>>
>>> Route: /{tenant_id}/capabilities/
>>>
>>> CLI call: trove capabilities-list
>>> Capabilities Management API
>>>
>>>
>>>
>>> My suggestion is to add capabilities management API to trove-manageutility (for the first iteration).
>>>
>>> trove-manage capabilities_update <datastore_manager>
>>> <Trove-”API-section”> <method> <Identifier>
>>>
>>> where:
>>>
>>> <datastore_manager> - datastore manager (variants: mysql, redis,
>>> cassandra, mongo, percona);
>>>
>>> <Trove-”API-section”> - part of API (core or extension, possible
>>> variants: Trove-Instance, Trove-Backup, Trove-Users, etc);
>>>
>>> <method> - method available for given section of Trove API (variants:
>>> create, delete, list, show etc)
>>>
>>> <Identifier> - identifier that marks method from API section as
>>> ALLOWED/BLOCKED
>>>
>>>
>>> [1] https://wiki.openstack.org/wiki/Trove/trove-capabilities
>>>
>>> [2] https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix
>>>
>>> [3]
>>> https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix#API_Matrix
>>>
>>> [4]
>>> https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix#Extensions_Matrix
>>>
>>>
>>>
>>> Best regards
>>>
>>> Denis Makogon.
>>> <https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix#Extensions_Matrix>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140226/252a297e/attachment-0001.html>
More information about the OpenStack-dev
mailing list