[openstack-dev] [all] REST API style guide (Re: [Nova] Some ideas for micro-version implementation)

Kenichi Oomichi oomichi at mxs.nes.nec.co.jp
Wed Sep 24 04:03:08 UTC 2014


> -----Original Message-----
> From: Alex Xu [mailto:xuhj at linux.vnet.ibm.com]
> Sent: Tuesday, September 23, 2014 12:20 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [all] REST API style guide (Re: [Nova] Some ideas for micro-version implementation)
> 
> >>> Before discussing how to implement, I'd like to consider what we should
> >>> implement. IIUC, the purpose of v3 API is to make consistent API with the
> >>> backwards incompatible changes. Through huge discussion in Juno cycle, we
> >>> knew that backwards incompatible changes of REST API would be huge pain
> >>> against clients and we should avoid such changes as possible. If new APIs
> >>> which are consistent in Nova API only are inconsistent for whole OpenStack
> >>> projects, maybe we need to change them again for whole OpenStack
> >>> consistency.
> >>>
> >>> For avoiding such situation, I think we need to define what is consistent
> >>> REST API across projects. According to Alex's blog, The topics might be
> >>>
> >>>   - Input/Output attribute names
> >>>   - Resource names
> >>>   - Status code
> >>>
> >>> The following are hints for making consistent APIs from Nova v3 API
> >>> experience,
> >>> I'd like to know whether they are the best for API consistency.
> >>>
> >>> (1) Input/Output attribute names
> >>> (1.1) These names should be snake_case.
> >>>    eg: imageRef -> image_ref, flavorRef -> flavor_ref, hostId -> host_id
> >>> (1.2) These names should contain extension names if they are provided in
> >>> case of some extension loading.
> >>>    eg: security_groups -> os-security-groups:security_groups
> >>>        config_drive -> os-config-drive:config_drive
> >>
> >> Do you mean that the os- prefix should be dropped? Or that it should be
> >> maintained and added as needed?
> > The above samples contain two meanings:
> >   - extension names are added. ("os-security-groups:", "os-config-drive:")
> >   - their extensions are not cores. (v3 ones should contain "os-")
> >
> > Their changes are Nova v3 API's ones, and now I have a question related to
> > your point.
> > Should we add extension names to each input/output attribute names?
> > How about naming them with snake_case only without extension names?
>
> I can think out of two purposes for add extension names to attribute
> names. One is used for namespace, Another one is used for distinguish
> between core and extension.

Is the namespace for avoiding duplicated attribute names?
Actually, I could not find duplicated names on current v2 api if removing
extension names from attributes.

> And with extension name is more readable for developer, developer can
> know this attribute come from which extension without search API doc.

Yeah, that is true. That is one merit of extension names.
but I'd like to know that is common thing on whole OpenStack projects.

> > To be honest, I'm not sure yet extension name values for each attribute name.
> > Additional extension names make attribute names long and complex.
> > In addition, if we define Pecan/WSME as standard web frameworks, we should name
> > attributes with snake_case only because of Pecan/WSME restriction[1]. So we can
> > not name them with hyphens and colons which are including in current extension
> > attribute names.
>
> yes, good point, that is problem. Is it only restriction for
> resource-name? "-"/":" works for getattr/setattr, it is a walk around,
> so I'm not sure it's worth.

The restriction works for both resource-name and attribute name also.

> >>> (1.3) Extension names should consist of hyphens and low chars.
> >>>    eg: OS-EXT-AZ:availability_zone ->
> >>> os-extended-availability-zone:availability_zone
> >>>        OS-EXT-STS:task_state -> os-extended-status:task_state
> >>
> >> Yes, I don't like the shoutyness of the ALL CAPS.
> >>
> >>> (1.4) Extension names should contain the prefix "os-" if the extension is
> >>> not core.
> >>>    eg: rxtx_factor -> os-flavor-rxtx:rxtx_factor
> >>>        os-flavor-access:is_public -> flavor-access:is_public (flavor-access
> >>> extension became core)
> >> Do we have a list of "core" yet?
> > We have the list in the code:
> >
> > nova/api/openstack/__init__.py
> >   62 # List of v3 API extensions which are considered to form
> >   63 # the core API and so must be present
> >   64 # TODO(cyeoh): Expand this list as the core APIs are ported to V3
> >   65 API_V3_CORE_EXTENSIONS = set(['consoles',
> >   66                               'extensions',
> >   67                               'flavor-extra-specs',
> >   68                               'flavor-manage',
> >   69                               'flavors',
> >   70                               'ips',
> >   71                               'os-keypairs',
> >   72                               'os-flavor-access',
> >   73                               'server-metadata',
> >   74                               'servers',
> >   75                               'versions'])
> >
> > # As you see, some extensions in the above list contains "os-" even if cores
> > # because we have reverted them to v2 names through v2.1 API development.
> >
> > but here also I have a question.
> > Is it necessary to define core by itself?
> > IIUC, DefCore is trying to define the core features of OpenStack and the tool
> > RefStack verifies these core behaviors through REST API.
> > Will they conflict in the future?
> >
> >>> (1.5) The length of the first attribute depth should be one.
> >>>    eg: "create a server" API with scheduler hints
> >>>      -- v2 API input attribute sample
> >>> ---------------------------------------
> >>>        {
> >>>            "server": {
> >>>                "imageRef": "e5468cc9-3e91-4449-8c4f-e4203c71e365",
> >>>                [..]
> >>>            },
> >>>            "OS-SCH-HNT:scheduler_hints": {
> >>>                "same_host": "5a3dec46-a6e1-4c4d-93c0-8543f5ffe196"
> >>>            }
> >>>        }
> >>>      -- v3 API input attribute sample
> >>> ---------------------------------------
> >>>        {
> >>>            "server": {
> >>>                "image_ref": "e5468cc9-3e91-4449-8c4f-e4203c71e365",
> >>>                [..]
> >>>                "os-scheduler-hints:scheduler_hints": {
> >>>                    "same_host": "5a3dec46-a6e1-4c4d-93c0-8543f5ffe196"
> >>>                }
> >>>            }
> >>>        }
> >>>
> >>> (2) Resource names
> >>> (2.1) Resource names should consist of hyphens and low chars.
> >>>    eg: /os-instance_usage_audit_log -> /os-instance-usage-audit-log
> >>> (2.2) Resource names should contain the prefix "os-" if the extension is
> >>> not core.
> >>>    eg: /servers/diagnostics -> /servers/os-server-diagnostics
> >>>        /os-flavor-access -> /flavor-access (flavor-access extension became
> >>> core)
> >>> (2.3) Action names should be snake_case.
> >>>    eg: os-getConsoleOutput -> get_console_output
> >>>        addTenantAccess -> add_tenant_access, removeTenantAccess ->
> >>> remove_tenant_access
> > The above resource names also are against Pecan/WSME restrictions
> > because they contain hyphens.
> >
> >>> (3) Status code
> >>> (3.1) Return 201(Created) if a resource creation/updating finishes before
> >>> returning a response.
> >>>    eg: "create a keypair" API: 200 -> 201
> >>>        "create an agent" API: 200 -> 201
> >>>        "create an aggregate" API: 200 -> 201
> >> Do you mean a 200 becomes a 201? That's a huge doc impact and SDK impact, is
> >> it worthwhile? If we do this change, the sooner the better, right?
> > No, these changes were v3 ones only, not current v2 API.
> > v3 API interfaces have disappeared now, and we will not change them without any
> > notifications(including Docs) because of backwards compatibilities.
> > So they don't affect to the existing SDKs and users.
> >
> > The reason why I picked them up is that I hope we consider what is the best REST
> > API design. On the above samples, Nova v3 API was trying to return more accurate
> > status codes for showing each resource internal status.
> >
> > I have a concern about this. If internal implementation of OpenStack
> > will be changed
> > like sync operation to async, should we change the status code also
> > for returning
> > accurate status code? That would make backwards incompatibilities. but
> > if not doing
> > that, status codes don't show accurate internal status.
>
> The backwards-incompatibility already happened when the internal
> implementation is changed like sync operation to async, this is kind of
> semantic change. This shouldn't happened.

As you said, maybe we can say "the backwards-incompatibility never happens"
only for APIs which are changed from sync to async. but we can do that for
the other APIs because we are not sure this kind of change never happen
forever.

> > So how about just using HTTP 200(OK) only for status codes?
> > That would give up providing accurate internal status to clients but backwards
> > backwards incompatibilities never happen.
> > # just one stupid idea :)
>
> I don't think this is right way. Client need to know whether the
> operation is finished or not.

Ok, I was too scared at backwards incompatible changes.

> >>> (3.2) Return 204(No Content) if a resource deletion finishes before
> >>> returning a response.
> >>>    eg: "delete a keypair" API: 200 -> 204
> >>>        "delete an agent" API: 200 -> 204
> >>>        "delete an aggregate" API: 200 -> 204
> >>> (3.3) Return 202(Accepted) if a request doesn't finish yet before
> >>> returning a response.
> >>>    eg: "rescue a server" API: 200 ->202
> >>>
> >> Same here, sooner the better if these are better response codes.
> > The same as the above (3.1).
> 
> One more thing for status code is error status code. We can't get
> consistent for all the case. But at least we can get consistent for the
> some usual cases, like over quota.

Yes, right. Error codes also should be consistent, but we can change them
if we find more accurate code without considering backwards incompatibilities
according to API change guidline.

https://wiki.openstack.org/wiki/APIChangeGuidelines

> >>> Any comments are welcome.
> >>>
> >> The TC had an action item a while back (a few months) to start an API style
> >> guide. This seems like a good start. Once the questions are discussed I'll
> >> get a draft going on the wiki.
> > Thanks again.
> > The REST API style guide would be nice for whole OpenStack projects.
> +1, thanks for writing this up!
> >
> > and I have one more idea for making API consistency of whole OpenStack projects.
> > That is each rule of the style guide is implemented in Tempest.
> > Tempest has its own REST clients for many projects and we can customize them
> > for improving qualities. After defining the REST API style guide, we
> > can add each
> > rule to Tempest's base client class and apply it for all REST APIs
> > which are tested
> > by Tempest. We can keep consistent API for the existing projects and apply the
> > style guide to new projects also by this framework.
>
> Do you mean some tests for API style? Some automatic test is always
> good! If it just a style checking, I'm not sure it's worth going to
> tempest, maybe just a tool that running
> before developer submit patch?

QA team has a plan[1] for moving api tests from Tempest to each projects
and tempest-lib is been developed now as the first step. tempest-lib will
be used for Tempest and projects for operating tests. If adding the above
style checking into tempest-lib, we will be able to check the style as unit
tests of each project.
If concentrating on Nova only, we could add the style checking to the base
part of sample tests. However, that way works for Nova only, because these
tests are Nova specific. So I feel now Tempest is a nice place for adding
this kind of check.

[1]: http://lists.openstack.org/pipermail/openstack-dev/2014-August/042601.html

Thanks
Ken'ichi Ohmichi



More information about the OpenStack-dev mailing list