Open Stack

On Wed, Oct 16, 2013 at 1:19 PM, Ken'ichi Ohmichi <ken1ohmichi at gmail.com>wrote:

> Hi Chris,
>
> Thank you for your response.
>
> 2013/10/16 Christopher Yeoh <cbkyeoh at gmail.com>
>
>> On Tue, Oct 15, 2013 at 5:44 PM, Kenichi Oomichi <
>> oomichi at mxs.nes.nec.co.jp> wrote:
>>
>>>
>>> Hi,
>>>
>>> # I resend this because gmail distinguished my previous mail as spam
>>> one..
>>>
>>> I'd like to know what validation feature is really needed for Nova v3
>>> API,
>>> and I hope this mail will be a kick-off of brain-storming for it.
>>>
>>> << Introduction >>
>>> I have submitted a blueprint "nova-api-validation-fw".
>>> The purpose is comprehensive validation of API input parameters.
>>> 32% of Nova v3 API parameters are not validated with any ways[1], and the
>>> fact would cause an internal error if some clients just send an invalid
>>> request. If an internal error happens, the error message is output to a
>>> log file and OpenStack operators should research its reason. It would be
>>> hard work for the operators.
>>>
>>>
>> We have tried to improve this for the V3 API but we still have a way to
>> go. I believe a validation framework like you have proposed would be very
>> useful - and cleanup the extension code.
>>
>
> I'm really glad about this comment :-)
>
>
>  In Havana development cycle, I proposed the implementation code of the BP
>>> but it was abandoned. Nova web framework will move to Pecan/WSME, but my
>>> code depended on WSGI. So the code would have merits in short term, but
>>> not
>>> in long term.
>>> Now some Pecan/WSME sessions are proposed for Hong-Kong summit, so I feel
>>> this situation is a good chance for this topic.
>>>
>>>
>> I proposed the Nova Pecan/WSME session for the summit, but I do have a
>> few reservations about whether the transition will be worth the pain I
>> think will be involved. So I don't think its by any means clear that
>> Pecan/WSME will be something we will do in Icehouse and your wsgi based
>> implementation could be what we want to go ahead with.
>>
>>
>>> For discussing, I have investigated all validation ways of current Nova
>>> v3
>>> API parameters. There are 79 API methods, and 49 methods use API
>>> parameters
>>> of a request body. Totally, they have 148 API parameters. (details: [1])
>>>
>>> Necessary features, what I guess now, are the following:
>>>
>>> << Basic Validation Feature >>
>>> Through this investigation, it seems that we need some basic validation
>>> features such as:
>>> * Type validation
>>>   str(name, ..), int(vcpus, ..), float(rxtx_factor), dict(metadata, ..),
>>>   list(networks, ..), bool(conbine, ..), None(availability_zone)
>>> * String length validation
>>>   1 - 255
>>> * Value range validation
>>>   value >= 0(rotation, ..), value > 0(vcpus, ..),
>>>   value >= 1(os-multiple-create:min_count, os-multiple-create:max_count)
>>> * Data format validation
>>>   * Pattern:
>>>     uuid(volume_id, ..), boolean(on_shared_storage, ..),
>>> base64encoded(contents),
>>>     ipv4(access_ip_v4, fixed_ip), ipv6(access_ip_v6)
>>>   * Allowed list:
>>>     'active' or 'error'(state), 'parent' or 'child'(cells.type),
>>>     'MANUAL' or 'AUTO'(os-disk-config:disk_config), ...
>>>   * Allowed string:
>>>     not contain '!' and '.'(cells.name),
>>>     contain [a-zA-Z0-9_.- ] only(flavor.name, flavor.id)
>>> * Mandatory validation
>>>   * Required: server.name, flavor.name, ..
>>>   * Optional: flavor.ephemeral, flavor.swap, ..
>>>
>>>
>>> << Auxiliary Validation Feature >>
>>> Some parameters have a dependency between other parameter.
>>> For example, name or/and availability_zone should be specified when
>>> updating an
>>> aggregate. The parameter dependencies are few cases, and the dependency
>>> validation
>>> feature would not be mandatory.
>>>
>>> The cases are the following:
>>> * Required if not specifying other:
>>>   (update aggregate: name or availability_zone), (host: status or
>>> maintenance_mode),
>>>   (server: os-block-device-mapping:block_device_mapping or image_ref)
>>> * Should not specify both:
>>>   (interface_attachment: net_id and port_id),
>>>   (server: fixed_ip and port)
>>>
>>>
>> These all sound useful.
>>
>>
>>> << API Documentation Feature >>
>>> WSME has a unique feature which generates API documentations from source
>>> code.
>>> The documentations(
>>> http://docs.openstack.org/developer/ceilometer/webapi/v2.html)
>>> contains:
>>> * Method, URL (GET /v2/resources/, etc)
>>> * Parameters
>>> * Reterun type
>>> * Parameter samples of both JSON and XML
>>>
>>
>> Do you know if the production of JSON/XML samples and integration of them
>> into the api documentation
>> is all autogenerated via wsme?
>>
>
> I'm not good at this feature.
> but Ceilometer's document(
> http://docs.openstack.org/developer/ceilometer/webapi/v2.html) would be
> generated from
> https://github.com/openstack/ceilometer/blob/master/ceilometer/api/controllers/v2.py#L891
>  etc.
> API samples also would be autogenerated from sample() method.
> I hope some experts will help us about this feature.
>

The ceilometer developer docs are processed with sphinx, and the sample
objects are turned from Python objects to XML or JSON and output in the
HTML by sphinxcontrib-pecanwsme (naming packages is hard :-). There is also
some work going on to generate the samples in the format that the doc team
uses for the public documentation (https://review.openstack.org/#/c/45592/),
although it doesn't look like it is quite complete because the argument
types aren't defined. I haven't seen the tool that is generating the WADL,
yet.

Doug


>
> The API sample implementations of Nova v3 API seemed very hard processes
> in Havana cycle.
> There are a lot of API parameters, and some API parameter names are
> renamed to right ones.
> Reviewers should check their name and the data types of API samples.
> I feel this feature is good for Nova v3 API, but I'm not sure whether we
> should replace with Pecan/WSME in Icehouse.
>
>
> Thanks
> Ken'ichi Ohmichi
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131016/b563a215/attachment-0001.html>