Open Stack

On 03/05/2014 10:25 PM, Christopher Yeoh wrote:
> On Tue, 04 Mar 2014 13:31:07 -0500
> David Kranz <dkranz at redhat.com> wrote:
>> I think it would be a good time to have at least an initial
>> discussion about the requirements for theses schemas and where they
>> will live. The next step in tempest around this is to replace the
>> existing negative test files with auto-gen versions, and most of the
>> work in doing that is to define the schemas.
>>
>> The tempest framework needs to know the http method, url part,
>> expected error codes, and payload description. I believe only the
>> last is covered by the current nova schema definitions, with the
>> others being some kind of attribute or data associated with the
>> method that is doing the validation. Ideally the information being
>> used to do the validation could be auto-converted to a more general
>> schema that could be used by tempest. I'm interested in what folks
>> have to say about this and especially from the folks who are core
>> members of both nova and tempest. See below for one example (note
>> that the tempest generator does not yet handle "pattern").
>
> So as you've seen a lot of what is wanted for the tempest framework is
> implicitly known already within the method context which is why its not
> again explicitly stated in the schema. Not actually having thought
> about it a lot, but I suspect the expected errors decorator is
> something that would fit just as well in the validation framework
> however.
>
> Some of the other stuff such as url part, descriptions etc, not so much
> as it would be purely duplicate information that would get out of date.
> However for documentation auto generation it is something that we do
> also want to have available in an automated fashion.  I did a bit of
> exploration early in Icehouse in generating this within the context of
> api samples tests where we have access to this sort of stuff and I think
> together we'd have all the info we need, I'm just not sure mashing them
> together is the right way to do it.
To be clear, I was advocating for some solution where all the 
information needed to create negative tests, api docs, (part of client 
libraries?), etc. could be derived from the source code. The pieces of 
information could come from explicit schema definitions, scanning the 
code, python introspection. I have done this sort of thing in other 
languages but am not enough of a Python wiz to say exactly how this 
could work. Certainly the schema should not have duplicate information.
> And from the documentation point of view we need to have a bit of a
> think about whether doc strings on methods should be the canonical way
> we produce descriptional information about API methods. One hand its
> appealing, on the other hand they tend to be not very useful or very
> internals Nova focussed. But we could get much better at it.
Agreed. IMO, doc strings on methods have far more benefits (automation, 
not getting out of sync with code) than real costs (non-coders have to 
edit the code to review or improve the docs). This works best when 
developers write at least drafts of doc strings. The REST API case is a 
little more tricky because it does not map as directly to methods.

  -David
>
> Short version - yea I think we want to get to the point where tempest
> doesn't generate these manually. But I'm not sure about how we
> should do it.
>
> Chris
>
>>    -David
>>
>>   From nova:
>>
>> get_console_output = {
>>       'type': 'object',
>>       'properties': {
>>           'get_console_output': {
>>               'type': 'object',
>>               'properties': {
>>                   'length': {
>>                       'type': ['integer', 'string'],
>>                       'minimum': 0,
>>                       'pattern': '^[0-9]+$',
>>                   },
>>               },
>>               'additionalProperties': False,
>>           },
>>       },
>>       'required': ['get_console_output'],
>>       'additionalProperties': False,
>> }
>>
>>   From tempest:
>>
>> {
>>       "name": "get-console-output",
>>       "http-method": "POST",
>>       "url": "servers/%s/action",
>>       "resources": [
>>           {"name":"server", "expected_result": 404}
>>       ],
>>       "json-schema": {
>>           "type": "object",
>>           "properties": {
>>               "os-getConsoleOutput": {
>>                   "type": "object",
>>                   "properties": {
>>                       "length": {
>>                           "type": ["integer", "string"],
>>                           "minimum": 0
>>                       }
>>                   }
>>               }
>>           },
>>           "additionalProperties": false
>>       }
>> }
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev