Open Stack

Hi all,

According to our discussion on PTG and recent discussion by mailing list.  We have an agreement on using wiki to store the test report for the device drivers in the case that they do not have 3rd Party CI at present. 

Please see the wiki page here: https://wiki.openstack.org/wiki/Cyborg/TestReport.
Currently, there is one test report, other contributor who wants to upstream a device driver in Cyborg and who do not have the condition to hold a 3rd party CI can refer to this test report and give us your report when upstreaming.  Reference: https://wiki.openstack.org/wiki/Cyborg/TestReport/IntelQAT 

Thanks,
Xin-Ran

-----Original Message-----
From: Brin Zhang(张百林) <zhangbailin at inspur.com> 
Sent: Saturday, July 11, 2020 9:42 AM
To: smooney at redhat.com; yumeng_bao at yahoo.com; openstack-discuss at lists.openstack.org
Subject: 答复: [cyborg] Temporary treatment plan for the 3rd-party driver

On Fri, 2020-07-10 at 13:37 +0800, yumeng bao wrote:
> Brin, thanks for bringing this up!
> 
> > Hi all：
> >        This release we want to introduce some 3rd party drivers 
> > (e.g. Intel QAT, Inspur FPGA, and Inspur SSD etc.) in Cyborg, and we discussed the handling of 3rd-party driver CI in Cyborg IRC meeting [1].
> >        Due to the lack of CI test environment supported by hardware, 
> > we reached a temporary solution in two ways, as
> > follows:
> > 1. Provide a CI environment and provide a tempest test for Cyborg, 
> > this method is recommended; 2. If there is no CI environment, please 
> > provide the test results of this driver in the master branch or in 
> > the designated branch, which should be as complete as possible, sent to the Cyborg team, or pasted in the implementation of the commit.
> 
> Providing test result can be our option. The test result can be part 
> of the driver documentation[0] as this is public to users.
> And from my understanding, the test result should work as the role of 
> tempest case and clarify at least: necessary configuration,test operations and test results.

> i would advise against including the resulsts in docuemntation add int test results to a commit or provideing tiem at the poitn it merged just tells you it once worked on the developers system likely using devstack to deploy. it does not tell you that it still work after even a singel addtional commit has been merged. so i would sugges not adding the results to the docs as they will get out dateded quickly.

Good advice, this is also my original intention. Give the result verification in the submitted commit, and do not put the test verification result in the code base. As you said, this does not mean that it will always work unless a test report can be provided regularly. Of course, it is better if there is a third-party CI , we will try our best to fight for it.

> maintaining a wiki is fine but i woudl suggest considring any driver that does not have first or thirdparty ci to be experimental. the generic mdev driver we talked about can be tested using sampel kernel modules that provide realy mdevs implemnetaion of srial consoles or graphics devices. so it could be validated in first party ci and consider supported/non experimaental. if other driver can similarly be tested with virtual hardware or sample kernel modules that allowed testing in the first party ci they could alos be marked as fully supported. with out that level of testing however i would not advertise a driver as anything more then experimental.

> the old rule when i started working on openstack was if its not tested in ci its broken.


> 
> [0]
> https://docs.openstack.org/cyborg/latest/reference/support-matrix.html
> #driver-support
> 
> 
> >       [1]
> > http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openst
> > ack_cyborg.2020-07-02-03.05.log.html
> 
> Regards，
> Yumeng
>