Open Stack

I agree with Justin on 1.  JSON may be acceptable, but I don't believe we are validating throughly.  For example, in the JSON below..."active" is not a valid status, flavorId should be an integer,  etc.  That said, the JSON seems to be pretty close.  The XML is not that far behind it has the same problem as the JSON plus a missing namespace and some bad elements. Personally, I think we may be able to fix the XML with some creative WSGI and XSLT hacking but I'm not sure how easy it would be incorporating that into the code.

We really need to be writing tests around this stuff.  What we do in Rackspace today is that we generate the example in the DevGuide in XML validate the XML against the schema then translate to JSON,  then compare the translated  JSON against the JSON example in the DevGuide.   OR we take the JSON translate it to XML and validate the XML against the schema AND compare the XML against the example in the DevGuide...for instances where we expect input and output (servers, images), we iterate through multiple iterations of this.  Because there's a schema in the loop, we can catch errors like the ones I mentioned above even in the JSON.  Little validation errors are easy to sneak in and they do break clients.  The only real protection that we have is good testing.

As for the missing features in 1.0, I don't think that they're a big deal.  As Justin stated, the underlying implementation is missing AND they've been dropped from the 1.1 spec.  We should wait till the implementation is done -- I have a feeling they'll get implemented because I don't see Rackspace moving to Nova without them :-)   At that point, we can bring then in as extensions to 1.1, then put them into the core in a later version if we think they're worth it.

As for changes in 1.1.  I don't think we should be focused on maximizing compatibility when we switch from one version to another. The whole purpose of having different versions is that they introduce features that would otherwise break clients. That is client developers will have to go back to the code and make some changes in order to get things to work. If we always made compatible changes there would be no reason for changing the version.  I agree that we should be able to support at least two active versions at one time to give people a chance to migrate to the latest version.  I also think we should be investing in language bindings to help people along.   Let's talk about the changes that are coming in 1.1 and discuss exactly what we want those to look like in the summit.

Finally, I'd also rather XML support be marked as experimental in 1.0 rather than excluding it.  We have a large number of clients would rather use XML, because it's more testable or because they have better tooling.

-jOrGe W.


On Mar 31, 2011, at 7:24 PM, Gabe Westmaas wrote:

> Thanks Justin, appreciate the input!  Answers inline.
> 
> On Thursday, March 31, 2011 5:31pm, "Justin Santa Barbara" <justin at fathomdb.com> said:
> 
>> I think there are a few distinct issues:
>> 
>> 1) XML support.  I was thinking that most XML issues would also be issues in
>> the JSON, so validating the XML will also serve as validating the JSON.  I'd
>> appreciate an example here, but I agree in general that focusing on JSON is
>> acceptable - as long as its not just that we don't see the problems in JSON
>> because we're not validating it as thoroughly.
>> 
> 
> So the XML is generated based on the json, but it goes through an additional transformation so just checking the XML does not ensure that the json is correct.
> 
> Good point about an example, I should have provided one!  Below is the output for a JSON and XML request on the same resource (/servers/<id>).  Things are mostly correct until you get down to the IP address and metadata level.
> 
> {"server": 
>  {"status": "active", 
>   "hostId": "84fd63700cb981fed0d55e7a7eca3b25d111477b5b67e70efcf39b93", 
>   "addresses": {"public": [], "private": ["172.19.1.2"]}, 
>   "name": "metaserver", 
>   "flavorId": "m1.tiny", 
>   "imageId": "1", 
>   "id": 6, 
>   "metadata": {"data1": "value1"}
>  }
> }
> 
> <server flavorId="m1.tiny" hostId="84fd63700cb981fed0d55e7a7eca3b25d111477b5b67e70efcf39b93" id="6" imageId="1" name="metaserver" status="active">
>    <addresses>
>        <public/>
>        <private>
>            <item>
>                172.19.1.2
>            </item>
>        </private>
>    </addresses>
>    <metadata>
>        <data1>
>            value1
>        </data1>
>    </metadata>
> </server>
> 
> Correct XML would be:
> <server flavorId="m1.tiny" hostId="84fd63700cb981fed0d55e7a7eca3b25d111477b5b67e70efcf39b93" id="6" imageId="1" name="metaserver" status="active">
>    <addresses>
>        <public/>
>        <private>
>            <ip addr="10.176.42.16"/>
>        </private>
>    </addresses>
>    <metadata>
>        <meta key="data1">value1</meta>
>    </metadata>
> </server>
> 
> 
>> 2) Missing features.  I don't think of this as an API issue, unless these
>> are supported features which we simply aren't exposing.  My understanding is
>> that it's the underlying support that's holding this back, not the API
>> itself.
> 
> As far as 1.1 goes, I think the backend features that we were waiting on are there now, but as we all had a bunch of late merges, we didn't have the time to implement the API side before feature freeze.  For 1.0 missing features...we honestly didn't look at those two (shared ips and backup scheduling) because we didn't feel they added a ton of value above above what was coming in the 1.1 spec. So I'm not sure if the necessary backends are there, though I would guess no.  Someone else chime in here if that's wrong!
> 
>> 
>> 3) The v1.1 schema changes.  To be honest, I'm still confused here.  I think
>> you want to maximize compatibility with Cloud Servers v1.0, so it sounds
>> like you would also want a minimal v1.1 that was just the bare minimum
>> additional features to support additional features in nova (e.g.
>> changePassword action)  I don't really understand where the 'nice to have'
>> (e.g. atom links) changes came from in that case.  I think everyone would
>> support a minimal set of changes to the CloudServers v1.0 API; even if
>> people would rather have their own OpenStack API long-term, supporting the
>> two existing cloud APIs is an obvious win.
>> 
>> 
>>> Given this, what makes the most sense to me is to focus on stability for
>> version 1.0 API excluding XML support.  The bindings that are out there have
>> strong support for the JSON data formats in v1.0.  As suggested, I think we
>> call the current mostly implemented 1.1 API experimental so as to give
>> access to any features that we need that are only in 1.1.
>> 
>> I think v1.0 is a good API, and has the advantage that a lot of existing
>> clients work with it.  I would personally like XML support to be in there,
>> but I understand that we need to make sacrifices, so accept that the focus
>> should be on JSON in v1.0.
>> 
>> I think calling v1.1 experimental and discussing it at the Design Summit is
>> the right thing to do.  If you need to extend v1.0 (e.g. to support
>> restartServer in "v1.0"), then I think you can make the call on
>> re-numberings.
>> 
>>> One point I would like to ask about is how important is XML to the 1.0
>> implementation?
>> 
>> I personally would like to see XML support shipped, even if marked
>> experimental and not 100%.  I had thought that getting the XML right would
>> mean we got the JSON right also, and it's easier to test the XML.  However,
>> I totally agree that JSON should be our top priority.  Given the time
>> situation, I think XML work should only be done to help the JSON (e.g.
>> adding namespaces to the XML lets us do a schema validation, which
>> indirectly helps us check the JSON => yes.  Fixing XML output where the JSON
>> output is already valid => not a priority)
>> 
>> 
>> Justin
>> 
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp