Open Stack

On 07/09/2013 11:24 AM, Joshua McKenty wrote:
> Monty, you missed a couple of elements of the original program
> definition - the mappings from the tempest based "scorecard" to specific
> versions of the API docs, and the availability of the tested and
> verified endpoints to "tools ecosystem" developers.

Yes, great catch.

> Also, I believe the definition of "compliance" rests with the Board and
> not the TC, despite some debate as to whether we're capable of handling it.

I think it's a collaboration. I think the board owns the definition of
compliance, but I think that the TC has got to own the technical
implementation of that definition, both in the tooling but also in the
technical assessment of the results of tests. I don't think we need to
get legalistic with this yet... let's get something in front of people,
then we can battle on who gets to own looking at test result matrixes. :)

> And I'll arm-wrestle you for the PTL slot ;)

We should do that on a web-case...

> PS - for the purposes of FITS, the "candidate cloud" can be either a
> public cloud, or a test environment for private cloud software.

Yes, I believe someone would need to be able to run a copy of this
locally as well.

> On Jul 9, 2013 8:03 AM, "Monty Taylor" <mordred at inaugust.com
> <mailto:mordred at inaugust.com>> wrote:
> 
>     Hey all,
> 
>     I'd like to propose an official program to the TC - refstack, a program
>     for verifying interoperability between implementations via FITS testing.
> 
>     Official Title: OpenStack Interoperability
>     Initial PTL: Monty Taylor <mordred at inaugust.com
>     <mailto:mordred at inaugust.com>>
>     Mission Statement:
>       Develop and maintain FITS testing and interoperability reporting for
>       OpenStack deployments and distributions.
> 
>     For a little background, the specific incarnation of the idea for this
>     came up at the last in-person Foundation Board meeting, and I think it's
>     a good idea. Also, it turns out there’s a reference in the logo
>     guidelines stating that any FITS defined by the TC and made available by
>     OpenStack needs to be passed before the logos can be used for a product.
> 
>     Follow on discussions were held at the summit, and a general approach of
>     basing the testing of clouds on tempest was agreed to. The general idea
>     is that for any given cloud, refstack will run tempest against the cloud
>     in a standard configuration (not re-configuring tempest on a per-cloud
>     basis) This will result in a set of passing and failing tests. That
>     output then needs to be processed and presented in a way that it can be
>     the basis of determining which elements are compliant and not. The
>     ultimate decisions around which elements would be 'required' to be a
>     compliant deployment would come back to the TC, and how that compliance
>     translates in to trademark usage goes back to the board. But it's the
>     job of refstack to be able to test the candidate cloud and report on its
>     actual capabilities.
> 
>     In conjunction with this a 'reference' deployment config is needed, so
>     that we can validate that our test is passable. At the moment I would
>     expect this to be the de-facto standard which is devstack, but that's
>     ultimately a happenstance and opportunistic config, and over time the
>     refstack program would be involved in helping to define a deployment
>     configuration or configurations that we as a community feel should
>     reasonably be expected to pass our own FITS testing.
> 
>     Expected outputs:
>      - A service for submission of cloud endpoint information for testing
>      - Processing of tempest output into a service compliance report
>      - One or more deployment configurations that can, themselves, pass the
>     testing 100%
> 
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>