Open Stack

Some thoughts of “Capability” use cases:
1. Nova-compute service can discover host capabilities and tag them into its compute-node resource provider record.
2. Other services such as ironic and cinder may want to manage its resource pool, and they can tag capabilities themselves to that pool.
3. Admin/user can register user-defined capabilities to resource providers (i.e. a pool or a host)
4. Capabilities are managed/grouped/abstracted by namespaces, and scheduler can make decisions based on either cap_names or cap_namespaces
5. Placement service DON’T have any specific knowledge of a capability, it only know the its name, namespaces, its relationship to resource providers. They are used for scheduling, capability management and report.
6. Placement service need to know where a capability comes from(user-defined, nova-defined, or others), so it can have modification control of capabilities, and it can list existing capabilities according to types.

I think the above is the normal use cases, please correct me if there are mistakes or add more items.


Regards,
-Yingxin

From: Alex Xu [mailto:soulxu at gmail.com]
Sent: Monday, July 11, 2016 11:22 PM
To: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org>
Cc: Bhandaru, Malini K <malini.k.bhandaru at intel.com>; Cheng, Yingxin <yingxin.cheng at intel.com>; Jin, Yuntong <yuntong.jin at intel.com>; Tan, Lin <lin.tan at intel.com>
Subject: Re: [Nova] [RFC] ResourceProviderTags - Manage Capabilities with ResourceProvider

Matt mentioned Aggregate in the scheduler meeting, that is good question and also reminder me I should explain the relationship between Aggregate and ResourceProviderTags.

The Aggregate is expected to keep as a tool for group the hosts, and just for group hosts. People used to manage Capabilities with Aggregates, they put the hosts with some kind of Capabilities into same Aggregates, and using the metadata to identify the Capabilities. But Aggregate with metadata is really not very easy to manage.

Thinking of the case:

Host1 have Capability1
Host2 have Capability1 and Capability2
Host3 have Capability2 and Capability3.

With this case, we needs 3 aggregates for each Capability: agg_cap1, agg_cap2, agg_cap3. Then we needs add hosts to the aggregate as below:

agg_cap1: host1, host2
agg_cap2: host2, host3
agg_cap3: host3

When there are more capabilities and more hosts which needs to manage, the mapping of hosts and aggregate will be more complicate. And there isn't a easy interface to let user to know the specific host have what kind of capabilities.

The ResourceProviderTags will be a substitution of Aggregate on manage capabilities. With tags, it won't generate a complex mapping.

For the same case, we just need to add tags on the ResourceProvider. The interface of tags is pretty easy, check out the api-wg guideline https://github.com/openstack/api-wg/blob/master/guidelines/tags.rst. And the query parameter made the management easy.

There are also have some user want to write script to manage the Capabilities. Thinking the aggregate, the script will be very hard due to manage the mapping between aggregates and hosts. The script will be very easy with tags, due to tags just a set of string.


2016-07-11 19:08 GMT+08:00 Alex Xu <soulxu at gmail.com<mailto:soulxu at gmail.com>>:
This propose is about using ResourceProviderTags as a solution to manage Capabilities (Qualitative) in ResourceProvider.
The ResourceProviderTags is to describe the capabilities which are defined by OpenStack Service (Compute Service,
Storage Service, Network Service etc.) and by users. The ResourceProvider provide resource exposed by a single
compute node, some shared resource pool or an external resource-providing service of some sort.  As such,
ResourceProviderTags is also expected to describe the capabilities of single ResourceProvider or the capabilities of
ResourcePool.

The ResourceProviderTags is similar with ServersTags [0] which is implemented in the Nova. The only difference is
that the tags is attached to the ResourceProvider. The API endpoint will be " /ResourceProvider/{uuid}/tags", and it
will follow the API-WG guideline about Tags [1].

As the Tags are just strings, the meaning of Tag isn't defined by Scheduler. The meaning of Tag is defined by
OpenStack services or Users. The ResourceProviderTags will only be used for scheduling with a ResourceProviderTags
filter.

The ResourceProviderTags is very easy to cover the cases of single ResourceProvider, ResourcePool and
DynamicResouces. Let see those cases one by one.

For single ResourceProvider case, just see how Nova report ComputeNode's Capabilities. Firstly,  Nova is expected
to define a standard way to describe the Capabilities which provided by Hypervisor or Hardware. Then those description
of Capabilities can be used across the Openstack deployment. So Nova will define a set of Tags. Those Tags should
be included with prefix to indicated that this is coming from Nova. Also the naming rule of prefix can be used to catalog
the Capabilities. For example, the capabilities can be defined as:

    COMPUTE_HW_CAP_CPU_AVX
    COMPUTE_HW_CAP_CPU_SSE
    ....
    COMPUTE_HV_CAP_LIVE_MIGRATION
    COMPUTE_HV_CAP_LIVE_SNAPSHOT
    ....

( The COMPUTE means this is coming from Nova. HW means this is hardware related Capabilities. HV means this is
 capabilities of Hypervisor. But the catalog of Capabilities can be discussed separated. This propose focus on the
 ResourceTags. We also have another idea about not using 'PREFIX' to manage the Tags. We can add attributes to the
 Tags. Then we have more control on the Tags. This will describe separately in the bottom. )

Nova will create ResourceProvider for the compute node, and report the quantitative stuff, and report capabilities
by adding those defined tags to the ResourceProvider at same time. Then those Capabilities are exposed by Nova
automatically.

The capabilities of ComputeNode can be queried through the API "GET /ResourceProviders/{uuid}/tags".

For the ResourcePool case, let us use Shared Storage Pool as example. The different Storage Pool may have
different capabilities. Maybe one of Pool are using SSD. For expose that Capability, admin user can do as below:

1. Define the aggregates
  $AGG_UUID=`openstack aggregate create r1rck0610`

2. Create resource pool for shared storage
  $RP_UUID=`openstack resource-provider create "/mnt/nfs/row1racks0610/" \
    --aggregate-uuid=$AGG_UUID`

3. Update the capacity of shared storage
  openstack resource-provider set inventory $RP_UUID \
    --resource-class=DISK_GB \
    --total=100000 --reserved=1000 \
    --min-unit=50 --max-unit=10000 --step-size=10 \
    --allocation-ratio=1.0

4. Add the Capabilities of shared storage
  openstack resource-provider add tags $RP_UUID --tag STORAGE_WITH_SSD

In this case, 'STORAGE_WITH_SSD' is defined by Admin user. This is the same with Quantitative, where there
isn't agent to report the Quantitative, neither the Qualitative.

This is also easy to cover the DynamicResource case. Thinking of Ironic, admin will create ResourcePool for
same hardware configuration bare-metal machines. Those machines will have the same set of capabilities. So
those capabilities will be added to the ResourcePool as tags, this is pretty same with SharedStoragePool case.

To expose cloud capabilities to users,  there is one more API endpoint 'GET /ResourceProviders/Tags'. User can
get all the tags. Then user can know what kind of Capabilities the cloud provides. The query parameter
will allow user to filter the Tags by the prefix rules.

This propose is intended to be a solution of managing Capabilities in the scheduler with ResourceProvider. But yes,
look at how Nova implement the manage of Capabilities, this is just part of solution. The whole solution still needs needs
other propose (like [2]) to describe how to model capabilities inside the compute node and propose (like [3]) to
describe how to request capabilities.

Manage Tags with attributes
=====================
As described above, we add prefix to Tags to mark which service this Tag is coming from and which catalog or
namespaces of Capabilities this Tags belongs to. An alternative idea is adding attributes to the Tags.
We can use one attribute tags to mark the origin of capabilities: Which service this capability comes from, or is it a
user-defined tag. Another idea is to add an attribute to control the tag visibility. Some of Tags may not expected to
be usable by normal users. The third possible attribute is to mark the namespace of this tag to help group tags.
For example the prefix "HV" and "HW" can be managed in this way.Then we can query/manage those tags very
easily from database by attributes.

Reference
========
   [0] http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/tag-instances.html
   [1] https://github.com/openstack/api-wg/blob/master/guidelines/tags.rst
   [2] https://review.openstack.org/286520
   [3] https://review.openstack.org/313784

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160712/37c49478/attachment.html>