Open Stack

Hi Brian, thanks for the response. Some comments inline :)

On 02/11/2015 09:57 AM, Brian Rosmaita wrote:
> On 2/9/15, 8:44 PM, "Joe Gordon" <joe.gordon0 at gmail.com
> <mailto:joe.gordon0 at gmail.com>> wrote:
>
>
>     On Mon, Feb 9, 2015 at 1:22 PM, Jay Pipes <jaypipes at gmail.com
>     <mailto:jaypipes at gmail.com>> wrote:
>
>         On 01/20/2015 10:54 AM, Brian Rosmaita wrote:
>
>             From: Kevin L. Mitchell [kevin.mitchell at rackspace.com
>             <mailto:kevin.mitchell at rackspace.com>]
>             Sent: Monday, January 19, 2015 4:54 PM
>
>                 When we look at consistency, we look at everything else
>                 in OpenStack.
>                   From the standpoint of the nova API (with which I am
>                 the most familiar),
>                 I am not aware of any property that is ever omitted from
>                 any payload
>                 without versioning coming in to the picture, even if its
>                 value is null.
>                 Thus, I would argue that we should encourage the first
>                 situation, where
>                 all properties are included, even if their value is null.
>
>
>             That is not the case for the Images API v2:
>
>             "An image is always guaranteed to have the following
>             attributes: id,
>             status, visibility, protected, tags, created_at, file and
>             self. The other
>             attributes defined in the image schema below are guaranteed to
>             be defined, but is only returned with an image entity if
>             they have
>             been explicitly set." [1]
>
>
>         This was a mistake, IMHO. Having entirely extensible schemas
>         means that there is little guaranteed consistency across
>         implementations of the API.
>
>
>     +1, Subtle hard to discover differences between clouds is a pain for
>     interchangeability.
>
>
> Jay and Joe, thanks for weighing in.  I’m still not convinced that the
> course taken in the Images v2 API was a mistake, though.  (I wasn’t
> involved in its initial design, so this isn’t personal, just curiosity.)
>   Here are a few reasons why, maybe someone can set me straight?
>
> (1) Leaving null elements out is parsimonious.
> As long as there’s a JSON schema, the client has a good idea what to
> expect.  If you include
>        “whatever”: null
> in the response, I don’t see what that buys you.  If you simply don’t
> include the “whatever” element, the recipient knows it’s not set.  If
> you do include it set to null, you know that it’s not set … and you
> increased the size of the response payload without increasing its
> informativeness.  Further, even if you include the “whatever” element
> set to null, the client is still going to have to check it to handle the
> null case, so it’s really just a matter of how the client checks, not
> whether it has to check.

Agreed, it doesn't buy you much at all. I'm more interested in just 
being consistent across APIs regarding this.

> (2) Leaving null elements out doesn’t affect interchangeability.
> If our convention is that unset elements aren’t included, and we’ve got
> a JSON schema, then everyone knows what’s up.  Further, looking
> specifically at the use cases for images in Glance, different clouds
> have different sets of image properties that they use for specific
> purposes that may be unique to their cloud.

And this, right here, is not something we should encourage. Tag images 
with whatever free-form tags you wish, as a user, but deployers of the 
Glance image service should be able to say attribute XYZ means the same 
thing across different deployments of Glance. Otherwise, there's no use 
to having those attributes, IMO, since you cannot rely on them meaning 
the same thing.

 >  For example, some may put a
> hyperlink to licensing info in an image property, or versioning info, or
> package lists, or whatever you can fit in 255 chars.  So a client
> (intelligent or not) connecting to various clouds can’t expect to find
> the same set of properties defined in every cloud (except for the ones
> guaranteed by contract, which are listed above).  Thus, you’re going to
> have to deal with the problem of non-existent elements when you get to
> the additionalProperties in JSON no matter what.  But as long as you
> know this, you’re OK.  I think it’s a much bigger problem when you’ve
> got a mixture of null, “”, {} and other ways of conveying empty elements
> in a response.  By simply leaving properties out, there’s no question
> that they’re not set.

I do not think that additionalProperties should ever be anything other 
than false for any public API.

> (3) A little consistency is a good thing.
> Jay mentions that having entirely extensible schemas means that there’s
> little guaranteed consistency across implementations of the API.  In the
> Images API v2 case, the schema isn’t entirely extensible, you can add
> string-valued additionalProperties.  So there’s that.  But the bigger
> picture is that we’re in at the infancy of clouds and cloud management,
> there’s no way we can anticipate the set of Image properties that will
> be sufficient for all deployers.

Yes, I believe we can, and should, endeavour to determine the set of 
Image attributes that will be sufficient for all deployers, and evolve 
the schema of an Image over time to account for that.

We haven't even made the attempt to do that. Instead, we punted and said 
"OK, we decided the easy part, put these attributes in here as 
'standard' and then just let everything else be a free-for-all."

 >  So as long as the consistency
> guarantees are met for the small set of properties they’re guaranteed
> for, I don’t have a problem with the majority of image properties being
> variable … as long as we know what type each is, which we do, they’re
> all strings.

If RAX has one of these free-form string properties called 
"hw_numa_cells" and HP uses the string "hw_numa_topology_cells", then 
boom, you've just created a maintenance nightmare and cross-cloud 
incompatibility.

This is why I support the types of patches like this one:

https://review.openstack.org/#/c/76234/25/nova/objects/image_meta_props.py

which attempt to catalog and standardize the free-form string attributes 
(which in Nova-land we like to call system metadata) into actual 
codified attributes of an Image object.

Note that I actually think that the above properties should just be 
attributes on an Image object, but the idea is the same.

Best,
-jay