----
On Tue, 25 Feb 2020 01:16:13 -0600 prakash RAMCHANDRAN <pramchan@yahoo.com>
wrote ----
Mark,
Glad you pointed to right code. Reviewed and stand corrected. My mis-underdtanding was I considered 20919.01 as first release and 2019.02 as second release of the year. However based on your comments and reference I understand that it is year.month of release
, thus 2019.11 includes 'usuri' and previous 3 releases as listed pointed by you.
"os_trademark_approval": {
"target_approval": "2019.11",
"replaces": "2019.06",
"releases": ["rocky", "stein", "train", "ussuri"],
"status": "approved"
}
},
That clears that I should have asked for 2019.11.
Few more questions on Tempedt tests.
I read some where that we have about 1500 Tempest tests overall. Is that correct?
Yeah,
it might be little more or less but around 1500 in-tree tests in Tempest.
The interop code lines have gone down from 3836 lines to 3232 in train to usuri.
Looks contrary to growth, any comments?
As Thierry pointed out, a chunk of this was due to the removal of the volumes-v2 API from the required list. A few other tests have been removed over time for various other reasons (changes to or removal of tests in Tempest, etc). Since the test lists
are kept in git, you can actually walk through the complete history yourself to see why some tests were added or removed if you’d like:
It’s also important to note that just because we’ve added more tests over time to Tempest, more projects over time to OpenStack, or more API’s to existing projects, that doesn’t mean there will be a corresponding increase in the number of tests required
for the trademark programs. The OpenStack Powered program is more a trailing indicator of adoption than an attempt to force commercial products to support any and all capabilities. Each API is considered for inclusion in the program against a set of criteria
detailed here:
So, for example: if a project introduced a new API in Usuri, it’s highly unlikely that it would appear in the very next Guideline because it would fail to meet several criteria. It would be unlikely to be “widely deployed” since many deployments would
still be using Stein or Train (also covered in the same Guideline). It might not yet be supported in many third-party SDK’s or tools, so it might not yet meet the “used by tools” criteria. It might be only supported by one or two plugins/drivers in it’s
first release. Etc, etc, etc. Over time that API might gain adoption, meet sufficient criteria, and be added to the required list--or it might not.
If you’re curious about the history of all this and the process, you might have a look at this slightly old but still mostly relevant deck:
Question then is 60 compute and 40 storage lines I see in test cases, do we have stats for Tempest tests what's the distribution of 1500 tests across Platform, compute, storage etc. Where and how can I get that. information as documented report.
Those
should be counted from interop guidelines where you have mapping of capabilities with test cases.
Few
or most of the capabilities have one test to verifying it or a few more than one test.
For
example, "compute-flavors-list" capability is verified by two tests[2].
This
way you can count and identify the exact number of tests per compute, storage etc.
If
you would like to know about the Tempest test categorization, you can find it from the directory
structure.
We have structured the tests service wise directory, for example, all compute tests under
tempest/api/compute
[2].
Based on above should we expect decrease or increase for say 2020.05 Vancouver release?
Anecdotally, the Guidelines haven’t been changing very much in recent times as the “core” capabilities that have met with a lot of adoption seem fairly well established (though there have been a few new ones added and a few removed). I wouldn’t expect
dramatic changes.
How does one certify a kubernetes cluster running openstak modules, one module per docker container in a kubrrnetes cluster using tempest say like in Airship Control Plane on k8s worker node, which is a OpenStack over kubernetes cluster.
Is this possible and if so what test we need to modify to test and certify a Containerized OpenStack in Airship as OpenStack Platform?
I
should be verified same way via Tempest. Tempest does not reply on how OpenStack is deployed it interacts via the public
interface
(where interop needs only user-facing API excluding admin API) of each service which should be accessible on
k8s
cluster also. Tests are not required to modify in this case.
NOTE:
we can always extend (writing new tests for current 6 services tests) Tempest for tests required for
interop
capabilities either that are new or improved in the verification of existing ones. I remember
about
the discussion on covering the API microversion. We keep adding new tests to cover the
new
microverison where we need integration tests but we can add API tests also if interop requires those.
Right on. In fact, the capabilities in the interop programs are intended to be usable independent of how a cloud is deployed (including whether it’s containerized or on bare metal, whether it uses KVM/OVS/Ceph or vSphere/NSX/vSAN, whether it’s highly
available or a single-box appliance, etc).
Can we even certify if for say 2019.11?
The OpenStack Foundation’s interoperability programs allow you to use either of two most recently approved Guidelines for your testing (which as of right now are 2019.11 and 2019.06). Once the Board of Directors approves the next guideline, 2019.06 will
rotate out. I should note though: the Foundation’s interoperability programs are primarily targeted at commercial products (distributions, public clouds, appliances, managed services, etc). There’s no real reason an open source product couldn’t use these
same tests of course!
At Your Service,
Mark T. Voelker
This should open up exciting possibilities if practical to extend OpeStack powered platform to Airship.
Like to hear anyone who has insight to educate us on that.
ThanksPrakash
Sent from Yahoo Mail on Android
On Mon, Feb 24, 2020 at 6:09 AM, Mark Voelker<mvoelker@vmware.com> wrote: Hi Prakash,
I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02.
Hm, there actually isn’t a 2019.02 guideline--were you perhaps referring to 2019.06 or 2019.11? 2019.06 does cover Train but not Usuri [1], 2019.11 covers both [2]. As an FYI, the OpenStack Marketplace does list which guideline a particular product was most
recently tested against (refer to https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=K9zbzRvFzs3cxVnCocij2Sjy43MNyejDKuRN7ivQkMk%3D&reserved=0 for
example, and look for the green “TESTED” checkmark and accompanying guideline version), though this obviously doesn’t tell you what testing might be currently in flight.
[1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.06.json%23L75%5B2&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561657063&sdata=Vcy8iL%2F8L93m0%2B3PIwikD5vN%2BaREJigrncz7cjd3Zyc%3D&reserved=0] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fbranch%2Fmaster%2F2019.11.json%23L75&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=pTrl8xMypZJAPFu2vzQwFFOWnJu51k44oxn2PrrQI2Y%3D&reserved=0
At Your Service,
Mark T. Voelker
On Feb 22, 2020, at 5:04 AM, prakash RAMCHANDRAN <pramchan@yahoo.com> wrote:
Hi all,
I am curious to find out if any Distribution or Products based on Openstack Train or Usuri are seeking the latest certifications based on 2019.02.
Similarly does any Hardware Driver of Software application seeking OpenStack compatibility Logo?
Finally does anyone think that Open Infra Distro like Airship or StarlingX should promote Open Infra Airship or Open Infra StarlingX powered as a new way to promote eco system surrounding them similar to OpenStack compatible drivers and software.
Will then Argo, customize,Metal3.io or Ironic be qualified as Open Infra Airship compatible?
If so how tempest can help in testing the above comments?
Refer to this market place below as how Distos and Products leverage OpenStack logos and branding programs.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fmarketplace%2Fdistros%2F&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=rABzR1tWXUHinSG4jpe46ayuELsini4fbKcJhsxrN9g%3D&reserved=0
Discussions and feedback are welcome. A healthy debate as how k8s modules used in Open Infra can be certified will be a good start.
ThanksPrakash
Sent from Yahoo Mail on Android
[1] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Finterop%2Fsrc%2Fcommit%2F8f2e82b7db54cfff9315e5647bd2ba3dd6aacaad%2F2019.11.json%23L260-L281&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=wncg32ql1Gq%2B%2F7fZPUDdPzSv2Fbay2zCAXPT%2F9%2BQIo4%3D&reserved=0
[2] https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopendev.org%2Fopenstack%2Ftempest%2Fsrc%2Fbranch%2Fmaster%2Ftempest%2Fapi%2Fcompute&data=02%7C01%7Cmvoelker%40vmware.com%7C5567a591df5a45322f6808d7ba037f79%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637182396561667051&sdata=mgDV%2Bng445lIrj3CjMwdTeifyGMdXyogQhnVdXk4FS0%3D&reserved=0
-gmann