[security-sig] Ask base recommendation for handling vulnerability which affects both OpenStack code and vendor code
Hi OpenStack community, I’m working on OpenStack project under my employee company. In OpenStack, few of projects (e.g. Cinder and Ironic) uses vendor supplied library in it. This means vendor-neutral base code of project is hosted under OpenStack project repository but vendor hardware specific logic is delegated to vendor library which is called by base code and is maintained by each hardware vendor under their repository. In the past, I faced situation to handle vulnerability in which cause of vulnerability is in vendor library but, to handle vulnerability, both OpenStack code and vendor library code should be modified. To handle this, I consulted community member and asked them to coordinate to fix vulnerability in private way (i.e. review patch and prepare commit in not public but private as usual security handling manner). However member are not willing to working on together because cause of vulnerability is in vendor provided library. I felt it’s better for community to, at least, be open to working on such vulnerability because that policy may benefit community. So I think it’s better to include policy in VMT document[1] like “Community is encouraged to make effort to be open to handling vulnerability in which both OpenStack community code and vendor provided code should be modified”. How do you think about this? Regards, Vanou Ishii [1] https://security.openstack.org/vmt-process.html
[I'm Cc'ing you since you don't appear to be subscribed to openstack-discuss, but please reply to the list rather than to me directly.] On 2023-04-18 12:42:46 +0000 (+0000), Vanou Ishii (Fujitsu) wrote: [...]
In the past, I faced situation to handle vulnerability in which cause of vulnerability is in vendor library but, to handle vulnerability, both OpenStack code and vendor library code should be modified. To handle this, I consulted community member and asked them to coordinate to fix vulnerability in private way (i.e. review patch and prepare commit in not public but private as usual security handling manner). However member are not willing to working on together because cause of vulnerability is in vendor provided library. I felt it’s better for community to, at least, be open to working on such vulnerability because that policy may benefit community. So I think it’s better to include policy in VMT document[1] like “Community is encouraged to make effort to be open to handling vulnerability in which both OpenStack community code and vendor provided code should be modified”.
How do you think about this? [...]
I think it depends a great deal on the specifics of the bug and the proposed solution. OpenStack has backport policies which attempt to avoid introducing regressions or disruptive behavior changes in stable branches, so there are limits as to what can be changed through the backporting process. For fixing in the version of OpenStack services under development (currently 2023.2 "bobcat"), there is more leniency but it's still up to the core reviewers of that project as to what sorts of solutions they're willing to incorporate into their software in order to cater to changes in externally-developed device drivers. It sounds like you have a security vulnerability in a device driver, and fixing it will also require some changes to an OpenStack service, is that correct? If so, follow the recommended reporting process[*] ideally opening a bug in Launchpad or creating a story in StoryBoard depending on where that project's defect tracking is managed. If the affected repository is not officially overseen[**] by the OpenStack Vulnerability Management Team, you can still explicitly subscribe them (~openstack-vuln-mgmt in Launchpad or openstack-security in StoryBoard) and we'll try to assist with coordination on a best effort basis. [*] https://security.openstack.org/#how-to-report-security-issues-to-openstack [**] https://security.openstack.org/repos-overseen.html -- Jeremy Stanley
Hi Jeremy, Thanks for reply. I have already fixed that vulnerability and fix patch was merged into OpenStack project. In that case, patch review was held in public even that patch was related to vulnerability fix. What I would like to say is following: "I think it's good idea to add policy to OpenStack security document[1]. That policy is OpenStack community is encouraged to be open to handle vulnerability which needs fix both in OpenStack code and vendor library code if that vendor library takes good role in that OpenStack project." Above idea comes because when I consulted OpenStack project member about vulnerability fix whose root cause is in vendor library but both OpenStack code and vendor library, member was not open to collaborate working on vulnerability. Is such addition to OpenStack security document reasonable? I ask this because there was discussion, in my colleague team, about there is no explicit description to openness to working on such situation in OpenStack doc. Regards, Vanou [1] https://security.openstack.org -----Original Message----- From: Jeremy Stanley <fungi@yuggoth.org> Sent: Tuesday, April 18, 2023 11:46 PM To: openstack-discuss@lists.openstack.org Cc: Ishii, Vanou/石井 伴旺 <ishii.vanou@fujitsu.com> Subject: Re: [security-sig] Ask base recommendation for handling vulnerability which affects both OpenStack code and vendor code [I'm Cc'ing you since you don't appear to be subscribed to openstack-discuss, but please reply to the list rather than to me directly.] On 2023-04-18 12:42:46 +0000 (+0000), Vanou Ishii (Fujitsu) wrote: [...]
In the past, I faced situation to handle vulnerability in which cause of vulnerability is in vendor library but, to handle vulnerability, both OpenStack code and vendor library code should be modified. To handle this, I consulted community member and asked them to coordinate to fix vulnerability in private way (i.e. review patch and prepare commit in not public but private as usual security handling manner). However member are not willing to working on together because cause of vulnerability is in vendor provided library. I felt it’s better for community to, at least, be open to working on such vulnerability because that policy may benefit community. So I think it’s better to include policy in VMT document[1] like “Community is encouraged to make effort to be open to handling vulnerability in which both OpenStack community code and vendor provided code should be modified”.
How do you think about this? [...]
I think it depends a great deal on the specifics of the bug and the proposed solution. OpenStack has backport policies which attempt to avoid introducing regressions or disruptive behavior changes in stable branches, so there are limits as to what can be changed through the backporting process. For fixing in the version of OpenStack services under development (currently 2023.2 "bobcat"), there is more leniency but it's still up to the core reviewers of that project as to what sorts of solutions they're willing to incorporate into their software in order to cater to changes in externally-developed device drivers. It sounds like you have a security vulnerability in a device driver, and fixing it will also require some changes to an OpenStack service, is that correct? If so, follow the recommended reporting process[*] ideally opening a bug in Launchpad or creating a story in StoryBoard depending on where that project's defect tracking is managed. If the affected repository is not officially overseen[**] by the OpenStack Vulnerability Management Team, you can still explicitly subscribe them (~openstack-vuln-mgmt in Launchpad or openstack-security in StoryBoard) and we'll try to assist with coordination on a best effort basis. [*] https://security.openstack.org/#how-to-report-security-issues-to-openstack [**] https://security.openstack.org/repos-overseen.html -- Jeremy Stanley
On 2023-04-19 14:18:32 +0000 (+0000), Vanou Ishii (Fujitsu) wrote: [...]
I have already fixed that vulnerability and fix patch was merged into OpenStack project. In that case, patch review was held in public even that patch was related to vulnerability fix.
Note that it's preferable for most vulnerability fixes to be developed in public, since private embargoes are extremely inconvenient and taxing on our community. Only the most severe of vulnerabilities need secrecy and rigorous coordinated disclosure. I like to refer folks to this article on "the hidden costs of embargoes" which does a great job of explaining the delicate balance we try to strike in deciding which process to follow for each bug: https://access.redhat.com/blogs/766093/posts/1976653
What I would like to say is following:
"I think it's good idea to add policy to OpenStack security document[1]. That policy is OpenStack community is encouraged to be open to handle vulnerability which needs fix both in OpenStack code and vendor library code if that vendor library takes good role in that OpenStack project."
Above idea comes because when I consulted OpenStack project member about vulnerability fix whose root cause is in vendor library but both OpenStack code and vendor library, member was not open to collaborate working on vulnerability.
Is such addition to OpenStack security document reasonable? I ask this because there was discussion, in my colleague team, about there is no explicit description to openness to working on such situation in OpenStack doc. [...]
As a community we expect our projects to be open to collaboration with everyone; it's one of our founding principles: https://governance.openstack.org/tc/reference/principles.html#contribution-i... Security vulnerabilities are just bugs, and our projects are expected to be open to collaborating with members of the community on bug fixes. However, individual contributors decide their priorities and get to choose what they spend their time working on. Just because one contributor wasn't interested in collaborating on a specific bug fix doesn't mean others won't assist. Reaching out to project leadership when you have concerns over prioritization is the most effective way to make progress. This doesn't really seem to me to have anything to do with security vulnerabilities, and so I don't think that adding collaboration recommendations to our vulnerability reporting guidelines or to our vulnerability coordination process document would make sense. These principles apply everywhere, not just to security fixes. -- Jeremy Stanley
On Wed, Apr 19, 2023 at 10:14 AM Vanou Ishii (Fujitsu) < ishii.vanou@fujitsu.com> wrote:
Hi Jeremy,
Thanks for reply. I have already fixed that vulnerability and fix patch was merged into OpenStack project. In that case, patch review was held in public even that patch was related to vulnerability fix. What I would like to say is following:
"I think it's good idea to add policy to OpenStack security document[1]. That policy is OpenStack community is encouraged to be open to handle vulnerability which needs fix both in OpenStack code and vendor library code if that vendor library takes good role in that OpenStack project."
Above idea comes because when I consulted OpenStack project member about vulnerability fix whose root cause is in vendor library but both OpenStack code and vendor library, member was not open to collaborate working on vulnerability. Is such addition to OpenStack security document reasonable? I ask this because there was discussion, in my colleague team, about there is no explicit description to openness to working on such situation in OpenStack doc.
I believe there is a difference of impression as well, which can impact perception of openness. An explicit statement or policy document requirement won't fix the impression or perception which may result. There can be agreement to fix an issue, but disagreement on the process or the constraints. For example, in OpenStack, we have specific policies to handle the behavior of backports in terms of breaking changes. It means for such fixes, we have a very narrow path we can walk, and the more consensus and community buy-in we generate in that process makes everyone's job easier. Yet, the original proposed fix may not meet the policy requirements and usage constraints, and that push back can leave an impression of refusal to collaborate, which is not the intent. The key is in a dialog. Dialog is often required and it requires mutual compromise and respect for boundaries. Depending on a project's context, it may be that what is being asked of them is bordering on a feature because an end user may still need to make a choice and the prior insecure path may still need to be supported. This all can leave differing impressions, which is definitely difficult to navigate and only compounded with competing demands on contributor's time. -Julia Regards,
Vanou
[1] https://security.openstack.org
-----Original Message----- From: Jeremy Stanley <fungi@yuggoth.org> Sent: Tuesday, April 18, 2023 11:46 PM To: openstack-discuss@lists.openstack.org Cc: Ishii, Vanou/石井 伴旺 <ishii.vanou@fujitsu.com> Subject: Re: [security-sig] Ask base recommendation for handling vulnerability which affects both OpenStack code and vendor code
[I'm Cc'ing you since you don't appear to be subscribed to openstack-discuss, but please reply to the list rather than to me directly.]
On 2023-04-18 12:42:46 +0000 (+0000), Vanou Ishii (Fujitsu) wrote: [...]
In the past, I faced situation to handle vulnerability in which cause of vulnerability is in vendor library but, to handle vulnerability, both OpenStack code and vendor library code should be modified. To handle this, I consulted community member and asked them to coordinate to fix vulnerability in private way (i.e. review patch and prepare commit in not public but private as usual security handling manner). However member are not willing to working on together because cause of vulnerability is in vendor provided library. I felt it’s better for community to, at least, be open to working on such vulnerability because that policy may benefit community. So I think it’s better to include policy in VMT document[1] like “Community is encouraged to make effort to be open to handling vulnerability in which both OpenStack community code and vendor provided code should be modified”.
How do you think about this? [...]
I think it depends a great deal on the specifics of the bug and the proposed solution. OpenStack has backport policies which attempt to avoid introducing regressions or disruptive behavior changes in stable branches, so there are limits as to what can be changed through the backporting process. For fixing in the version of OpenStack services under development (currently 2023.2 "bobcat"), there is more leniency but it's still up to the core reviewers of that project as to what sorts of solutions they're willing to incorporate into their software in order to cater to changes in externally-developed device drivers.
It sounds like you have a security vulnerability in a device driver, and fixing it will also require some changes to an OpenStack service, is that correct? If so, follow the recommended reporting process[*] ideally opening a bug in Launchpad or creating a story in StoryBoard depending on where that project's defect tracking is managed. If the affected repository is not officially overseen[**] by the OpenStack Vulnerability Management Team, you can still explicitly subscribe them (~openstack-vuln-mgmt in Launchpad or openstack-security in StoryBoard) and we'll try to assist with coordination on a best effort basis.
[*] https://security.openstack.org/#how-to-report-security-issues-to-openstack [**] https://security.openstack.org/repos-overseen.html -- Jeremy Stanley
participants (3)
-
Jeremy Stanley
-
Julia Kreger
-
Vanou Ishii (Fujitsu)