Re: [cyborg] Temporary treatment plan for the 3rd-party driver
Brin, thanks for bringing this up!
Hi all：
This release we want to introduce some 3rd party drivers (e.g. Intel QAT, Inspur FPGA, and Inspur SSD etc.) in Cyborg, and we discussed the handling of 3rd-party driver CI in Cyborg IRC meeting [1].
Due to the lack of CI test environment supported by hardware, we reached a temporary solution in two ways, as follows:
1. Provide a CI environment and provide a tempest test for Cyborg, this method is recommended;
2. If there is no CI environment, please provide the test results of this driver in the master branch or in the designated branch, which should be as complete as possible, sent to the Cyborg team, or pasted in the implementation of the commit.
Providing test result can be our option. The test result can be part of the driver documentation[0] as this is public to users. And from my understanding, the test result should work as the role of tempest case and clarify at least: necessary configuration,test operations and test results. [0] https://docs.openstack.org/cyborg/latest/reference/support-matrix.html#drive...
[1] http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openstack_cybo...
Regards， Yumeng
On Fri, 2020-07-10 at 13:37 +0800, yumeng bao wrote:
Brin, thanks for bringing this up!
Hi all： This release we want to introduce some 3rd party drivers (e.g. Intel QAT, Inspur FPGA, and Inspur SSD etc.) in Cyborg, and we discussed the handling of 3rd-party driver CI in Cyborg IRC meeting [1]. Due to the lack of CI test environment supported by hardware, we reached a temporary solution in two ways, as follows: 1. Provide a CI environment and provide a tempest test for Cyborg, this method is recommended; 2. If there is no CI environment, please provide the test results of this driver in the master branch or in the designated branch, which should be as complete as possible, sent to the Cyborg team, or pasted in the implementation of the commit.
Providing test result can be our option. The test result can be part of the driver documentation[0] as this is public to users. And from my understanding, the test result should work as the role of tempest case and clarify at least: necessary configuration,test operations and test results.
i would advise against including the resulsts in docuemntation add int test results to a commit or provideing tiem at the poitn it merged just tells you it once worked on the developers system likely using devstack to deploy. it does not tell you that it still work after even a singel addtional commit has been merged. so i would sugges not adding the results to the docs as they will get out dateded quickly. maintaining a wiki is fine but i woudl suggest considring any driver that does not have first or thirdparty ci to be experimental. the generic mdev driver we talked about can be tested using sampel kernel modules that provide realy mdevs implemnetaion of srial consoles or graphics devices. so it could be validated in first party ci and consider supported/non experimaental. if other driver can similarly be tested with virtual hardware or sample kernel modules that allowed testing in the first party ci they could alos be marked as fully supported. with out that level of testing however i would not advertise a driver as anything more then experimental. the old rule when i started working on openstack was if its not tested in ci its broken.
[0] https://docs.openstack.org/cyborg/latest/reference/support-matrix.html#drive...
[1] http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openstack_cybo...
Regards， Yumeng
On Fri, 2020-07-10 at 13:37 +0800, yumeng bao wrote:
Brin, thanks for bringing this up!
Hi all： This release we want to introduce some 3rd party drivers (e.g. Intel QAT, Inspur FPGA, and Inspur SSD etc.) in Cyborg, and we discussed the handling of 3rd-party driver CI in Cyborg IRC meeting [1]. Due to the lack of CI test environment supported by hardware, we reached a temporary solution in two ways, as follows: 1. Provide a CI environment and provide a tempest test for Cyborg, this method is recommended; 2. If there is no CI environment, please provide the test results of this driver in the master branch or in the designated branch, which should be as complete as possible, sent to the Cyborg team, or pasted in the implementation of the commit.
Providing test result can be our option. The test result can be part of the driver documentation[0] as this is public to users. And from my understanding, the test result should work as the role of tempest case and clarify at least: necessary configuration,test operations and test results.
i would advise against including the resulsts in docuemntation add int test results to a commit or provideing tiem at the poitn it merged just tells you it once worked on the developers system likely using devstack to deploy. it does not tell you that it still work after even a singel addtional commit has been merged. so i would sugges not adding the results to the docs as they will get out dateded quickly.
Good advice, this is also my original intention. Give the result verification in the submitted commit, and do not put the test verification result in the code base. As you said, this does not mean that it will always work unless a test report can be provided regularly. Of course, it is better if there is a third-party CI , we will try our best to fight for it.
maintaining a wiki is fine but i woudl suggest considring any driver that does not have first or thirdparty ci to be experimental. the generic mdev driver we talked about can be tested using sampel kernel modules that provide realy mdevs implemnetaion of srial consoles or graphics devices. so it could be validated in first party ci and consider supported/non experimaental. if other driver can similarly be tested with virtual hardware or sample kernel modules that allowed testing in the first party ci they could alos be marked as fully supported. with out that level of testing however i would not advertise a driver as anything more then experimental.
the old rule when i started working on openstack was if its not tested in ci its broken.
[0] https://docs.openstack.org/cyborg/latest/reference/support-matrix.html #driver-support
[1] http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openst ack_cyborg.2020-07-02-03.05.log.html
Regards， Yumeng
Hi all, According to our discussion on PTG and recent discussion by mailing list. We have an agreement on using wiki to store the test report for the device drivers in the case that they do not have 3rd Party CI at present. Please see the wiki page here: https://wiki.openstack.org/wiki/Cyborg/TestReport. Currently, there is one test report, other contributor who wants to upstream a device driver in Cyborg and who do not have the condition to hold a 3rd party CI can refer to this test report and give us your report when upstreaming. Reference: https://wiki.openstack.org/wiki/Cyborg/TestReport/IntelQAT Thanks, Xin-Ran -----Original Message----- From: Brin Zhang(张百林) <zhangbailin@inspur.com> Sent: Saturday, July 11, 2020 9:42 AM To: smooney@redhat.com; yumeng_bao@yahoo.com; openstack-discuss@lists.openstack.org Subject: 答复: [cyborg] Temporary treatment plan for the 3rd-party driver On Fri, 2020-07-10 at 13:37 +0800, yumeng bao wrote:
Brin, thanks for bringing this up!
Hi all： This release we want to introduce some 3rd party drivers (e.g. Intel QAT, Inspur FPGA, and Inspur SSD etc.) in Cyborg, and we discussed the handling of 3rd-party driver CI in Cyborg IRC meeting [1]. Due to the lack of CI test environment supported by hardware, we reached a temporary solution in two ways, as follows: 1. Provide a CI environment and provide a tempest test for Cyborg, this method is recommended; 2. If there is no CI environment, please provide the test results of this driver in the master branch or in the designated branch, which should be as complete as possible, sent to the Cyborg team, or pasted in the implementation of the commit.
Providing test result can be our option. The test result can be part of the driver documentation[0] as this is public to users. And from my understanding, the test result should work as the role of tempest case and clarify at least: necessary configuration,test operations and test results.
i would advise against including the resulsts in docuemntation add int test results to a commit or provideing tiem at the poitn it merged just tells you it once worked on the developers system likely using devstack to deploy. it does not tell you that it still work after even a singel addtional commit has been merged. so i would sugges not adding the results to the docs as they will get out dateded quickly.
Good advice, this is also my original intention. Give the result verification in the submitted commit, and do not put the test verification result in the code base. As you said, this does not mean that it will always work unless a test report can be provided regularly. Of course, it is better if there is a third-party CI , we will try our best to fight for it.
maintaining a wiki is fine but i woudl suggest considring any driver that does not have first or thirdparty ci to be experimental. the generic mdev driver we talked about can be tested using sampel kernel modules that provide realy mdevs implemnetaion of srial consoles or graphics devices. so it could be validated in first party ci and consider supported/non experimaental. if other driver can similarly be tested with virtual hardware or sample kernel modules that allowed testing in the first party ci they could alos be marked as fully supported. with out that level of testing however i would not advertise a driver as anything more then experimental.
the old rule when i started working on openstack was if its not tested in ci its broken.
[0] https://docs.openstack.org/cyborg/latest/reference/support-matrix.html #driver-support
[1] http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openst ack_cyborg.2020-07-02-03.05.log.html
Regards， Yumeng
Hi all. In today's IRC meeting [1], we decide to have a wiki to maintain the 3-rd-party drivers temporary test results, like Intel QAT driver test result [2], and we also need to maintain the Driver Support docs [3], add "Temporary Test Result" as a column in the Driver Support list, we should mark the result added time, such as the QAT driver result, may we can say "This test results reported at Aug. 2020 in Victoria Release, please reference https://wiki.openstack.org/wiki/Cyborg/TestReport/IntelQAT". In the Driver Support part, we will claim the "Temporary Test Result" is a temporary result, it will not always work. If you encounter problems during the adaptation process, please contact the Cyborg Core Team [4] for help. [1] http://eavesdrop.openstack.org/irclogs/%23openstack-cyborg/%23openstack-cybo... [2] https://wiki.openstack.org/wiki/Cyborg/TestReport/IntelQAT [3] https://docs.openstack.org/cyborg/latest/reference/support-matrix.html#drive... [4] https://review.opendev.org/#/admin/groups/1243,members brinzhang -----邮件原件----- 发件人: Brin Zhang(张百林) 发送时间: 2020年7月11日 9:41 收件人: 'smooney@redhat.com' <smooney@redhat.com>; 'yumeng_bao@yahoo.com' <yumeng_bao@yahoo.com>; 'openstack-discuss@lists.openstack.org' <openstack-discuss@lists.openstack.org> 主题: 答复: [cyborg] Temporary treatment plan for the 3rd-party driver On Fri, 2020-07-10 at 13:37 +0800, yumeng bao wrote:
Brin, thanks for bringing this up!
Hi all： This release we want to introduce some 3rd party drivers (e.g. Intel QAT, Inspur FPGA, and Inspur SSD etc.) in Cyborg, and we discussed the handling of 3rd-party driver CI in Cyborg IRC meeting [1]. Due to the lack of CI test environment supported by hardware, we reached a temporary solution in two ways, as follows: 1. Provide a CI environment and provide a tempest test for Cyborg, this method is recommended; 2. If there is no CI environment, please provide the test results of this driver in the master branch or in the designated branch, which should be as complete as possible, sent to the Cyborg team, or pasted in the implementation of the commit.
Providing test result can be our option. The test result can be part of the driver documentation[0] as this is public to users. And from my understanding, the test result should work as the role of tempest case and clarify at least: necessary configuration,test operations and test results.
i would advise against including the resulsts in docuemntation add int test results to a commit or provideing tiem at the poitn it merged just tells you it once worked on the developers system likely using devstack to deploy. it does not tell you that it still work after even a singel addtional commit has been merged. so i would sugges not adding the results to the docs as they will get out dateded quickly.
Good advice, this is also my original intention. Give the result verification in the submitted commit, and do not put the test verification result in the code base. As you said, this does not mean that it will always work unless a test report can be provided regularly. Of course, it is better if there is a third-party CI , we will try our best to fight for it.
maintaining a wiki is fine but i woudl suggest considring any driver that does not have first or thirdparty ci to be experimental. the generic mdev driver we talked about can be tested using sampel kernel modules that provide realy mdevs implemnetaion of srial consoles or graphics devices. so it could be validated in first party ci and consider supported/non experimaental. if other driver can similarly be tested with virtual hardware or sample kernel modules that allowed testing in the first party ci they could alos be marked as fully supported. with out that level of testing however i would not advertise a driver as anything more then experimental.
the old rule when i started working on openstack was if its not tested in ci its broken.
[0] https://docs.openstack.org/cyborg/latest/reference/support-matrix.html #driver-support
[1] http://eavesdrop.openstack.org/meetings/openstack_cyborg/2020/openst ack_cyborg.2020-07-02-03.05.log.html
Regards， Yumeng
participants (4)
-
Brin Zhang(张百林)
-
Sean Mooney
-
Wang, Xin-ran
-
yumeng bao