Open Stack

> From: Matthew Treinish [mtreinish at kortar.org]
> Sent: Tuesday, March 18, 2014 7:08 PM
> On Tue, Mar 18, 2014 at 12:34:57PM +0100, Koderer, Marc wrote:
> > On Tue, 18 Marc 2014 12:00:00 +0100
> > Christopher Yeoh [mailto:cbkyeoh at gmail.com] wrote:
> > > On Tue, 18 Mar 2014 10:39:19 +0100
> > > "Koderer, Marc" <m.koderer at telekom.de> wrote:
> > > >
> > > > I just recognized that we have very similar interface definitions in
> > > > tempest/api_schema and etc/schema:
> > > >
> > > > https://github.com/openstack/tempest/tree/master/etc/schemas
> > > > https://github.com/openstack/tempest/tree/master/tempest/api_schema
> > > >
> > > > Any objections if I move them to a single location? I'd prefer to use
> > > > json as file format instead of .py. As final goal we should find a way
> > > > how to merge them competently but I feel like this is something for
> > > > the design summit ;)
> > > >
> > >
> > > Heh we just moved them but I'm open to other suggestions - they are are
> > > specific to API testing though aren't they? Long term the idea is that
> > > they should be generated by Nova rather than tempest.  I think to prevent
> > > unintentional changes we'd probably cache a copy in tempest though rather
> > > than dynamically query them.
> 
> The idea was never to dynamically query them; there should always be a copy in
> the tempest tree. Like you said to prevent unintentional changes which is the
> same reason we don't auto-discover api versions. The idea for querying nova to
> get the schemas was to enable a tool which could populate the schemas
> automatically so that we didn't have to manually generate them individually. I'd
> say, to a certain extent, that this new round of validation patches could use
> the same kind of tool.
> 
> >
> > Sry that I didn't recognized this review.
> > Both definitions are coupled to API testing, yes.
> >
> > >
> > > My feeling at the moment is that they should  .py files.
> > > Because I think there's cases where we will want to have some schema
> > > definitions based on others or share common parts and use bits of python
> > > code to achieve this. For example availability zone list and detailed
> > > listing  have a lot in common (detailed listing just has a more
> > > parameters). I think there'll be similar cases for v2 and v3 versions as
> > > well.  While we're still manually generating them and keeping them up to
> > > date I think it's worth sharing as much as we can.
> >
> > Ok understood. We just converted the negative testing
> > definitions to json files due to review findings..
> 
> Well, when I left the review comment about it being a json file, I didn't think
> about inheritance. Chris has a good point about reusing common bits and just
> extending it. That wasn't how you proposed the negative test schemas would be
> used which is why I suggested using a raw json file.
> > It's just very confusing for new people if they see
> > two separate folders with schema definitions.
> >
> > But unfortunately there isn't an easy way.
> 
> Am I missing something or are these schemas being added now just a subset of
> what is being used for negative testing? Why can't we either add the extra
> negative test info around the new test validation patches and get the double
> benefit. Or have the negative test schemas just extend these new schemas being
> added?

Yes, the api_schema files should theoretically be a
subsets of the negative test schemas.
But I don't think that extending them will be possible:

if you have a property definition like this:

        "properties": {
            "minRam": {  "type": "integer",}

how can you extend it to:

        "properties": {
            "minRam": {
                "type": "integer",
                "results": {
                    "gen_none": 400,
                    "gen_string": 400
                }

This is the reason why I am unsure how inheritance can solve something here.

> >
> > >
> > > I agree there's a lot of commonality and we should long term just have one
> > > schema definition. There's quite a bit to discuss (eg level of strictness
> > > is currently different) in this area and a summit session about it would
> > > be very useful.
> > >
> >
> > +1
> >
> 
> I agree there is probably enough here to discuss during a summit session on
> where schema validation fits into tempest. As a part of that discussing how to
> store and manage schema definitions for both the negative test framework and
> validation tests.
> 
> 
> -Matt Treinish
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>