Hi, I’m reaching out to you to give you an update on the surveys that the OpenInfra Foundation community managers have been running. Our goal has been to understand challenges that contributors and maintainers in OpenStack have in order to improve the experience for both. To protect the anonymity of the survey respondents we are sharing anonymized and aggregated survey results reflecting feedback we received throughout OpenStack. As a next step we will reach out to project teams who received more than one survey response, starting with the team(s) that received most, to share team-specific feedback and understand how we can help to address them. Below please find highlights of the two surveys: __MAINTAINER SURVEY__ We received 15 responses from 11 project teams, with 36% of teams receiving more than one response. We asked respondents to rate the below statements between 1 (very bad) and 5 (excellent), which averaged at: - Code review - Changes you propose are reviewed in a timely manner: 2.73 - Code review - You receive actionable feedback from other reviewers: 4.26 - Code review - Automated test failures quickly direct you to problems in your changes: 3.6 - Contributor docs - It is comprehensive to cover processes and good practices: 3.73 - Contributor docs - It is up to date: 3.53 53% of responders said they didn’t refer to their projects Contributor Guide during the Epoxy release cycle, while 20% referred to it 1-9 times and 13% 10-19 times as well as 13% 20+ times. The survey also asked respondents to mark which listed issues they face while trying to land their own changes, as well as trying to land other contributors’ changes. Please note that respondents were allowed to choose as many options as were applicable to them. The below numbers are the percentages of respondents who faced each challenge: - Landing their own changes - Trouble getting review attention - 73% - Other - 33% - No changes pass CI - 13% - Need more assistance to proceed with review feedback - 6% - Expected to expand on review's scope - 6% - Disagreement or conflicting review comments - 6% - Landing other people’s changes - Change owner/stakeholders are unreachable to discuss change - 66% - Change owner can't/won't add missing pieces to the change - 66% - Change owner's response is slow - 60% - Change is lower priority than others - 40% - Change is beyond expertise - 40% - Change broken, doesn't pass CI - 40% - Testing is broken for the project - 20% - Other - 13% - Change needs a BP/spec - 6% The below numbers are the percentages of project teams who faced each challenge: - Landing their own changes - Trouble getting review attention - 64% - Other - 36% - No changes pass CI - 18% - Need more assistance to proceed with review feedback - 9% - Expected to expand on review's scope - 9% - Disagreement or conflicting review comments - 9% - Landing other people’s changes - Change owner/stakeholders are unreachable to discuss change - 82% - Change owner can't/won't add missing pieces to the change - 73% - Change owner's response is slow - 82% - Change is lower priority than others - 45% - Change is beyond expertise - 55% - Change broken, doesn't pass CI - 45% - Testing is broken for the project - 27% - Other - 18% - Change needs a BP/spec - 9% __CONTRIBUTOR SURVEY__ We received 28 responses from 15 project teams, with 53% of teams receiving more than one response. We asked respondents to rate the below statements between 1 (very bad) and 5 (excellent), which averaged at: - Contributor experience - Changes you propose are reviewed in a timely manner: 2.5 - Contributor experience - You receive actionable feedback from other reviewers: 4.25 - Contributor experience - Automated test failures quickly direct you to problems in your changes: 3.64 - Contributor documentation - You were able to find information about the processes the project team is using: 3.64 - Contributor documentation - It helped you to apply better practices throughout your contribution journey and achieve results faster: 3.57 - Contributor documentation - It is easy to discover: 3.32 - Contributor documentation - It is easy to navigate: 3.36 The survey also asked respondents to mark which listed issues they face while trying to land their own changes. Please note that respondents were allowed to choose as many options as were applicable to them. The below numbers are the percentages of respondents who faced each challenge: - Trouble getting review attention - 68% - Other - 36% - Unable to determine priorities - 18% - Reviewers keep coming back with new requests - 11% - Need more clarification on feedback to move forward - 7% - Reviewer expectation or consensus shifts over time - 7% - Expected to expand scope to address other project issues - 7% - Reviewers disagree or give conflicting feedback - 4% - Asked to make changes deviating from past consensus - 4% - Reviewer forgets to un-block change - 4% - Unhelpful or incorrect reviews - 4% The below numbers are the percentages of project teams who faced each challenge: - Trouble getting review attention - 73% - Other - 47% - Unable to determine priorities - 20% - Reviewers keep coming back with new requests - 20% - Need more clarification on feedback to move forward - 13% - Reviewer expectation or consensus shifts over time - 13% - Expected to expand scope to address other project issues - 13% - Reviewers disagree or give conflicting feedback - 7% - Asked to make changes deviating from past consensus - 7% - Reviewer forgets to un-block change - 7% - Unhelpful or incorrect reviews - 7% While most challenges are affecting some or many projects, the biggest issues across the board seem to be getting attention from reviewers, and having delays on both the reviewer and change owner side. This is not necessarily surprising knowing that many or rather most contributors and maintainers are working on OpenStack part-time, or even outside of their day jobs. With the limited bandwidth that the community has, prioritizing work items and clear communication are more important than ever for the success and sustainability of the project. OpenInfra community managers will start reaching out to project teams with most responses and start working on addressing the challenges they face. As we progress with next steps we will also share practices with the OpenStack community at large to help improve contributor experience for everyone. While the official survey deadline has passed, we are still looking for feedback. If you missed filling out the survey(s), please provide your input now! ~ Please fill out this survey separately for every OpenStack project you contributed to during the Epoxy release: https://openinfrafoundation.formstack.com/forms/openstack_contributor_satisf... ~ Please fill out this survey separately for every OpenStack project you were a core reviewer of during the Epoxy release: https://openinfrafoundation.formstack.com/forms/openstack_maintainer_satisfa... Thanks, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation
On Apr 2, 2025, at 07:27, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2025-04-01 19:49:13 +0000 (+0000), Jeremy Stanley wrote:
As the next step in this ongoing effort to improve the contributor and maintainer experience, community managers on the OpenInfra Foundation staff worked together to convert the pain points identified in prior phases into a pair of very quick (2-3 minute) anonymous surveys... [...]
Many thanks to Jens for pointing out in IRC that these are not entirely anonymous. The survey does require you to supply a name and E-mail address so that the foundation staff can reach out for clarification if needed on any free-form responses, and to help weed out bogus responses from bots or vandals.
Your responses are treated confidentially, however, as only foundation staff will have access to the raw response data which will be thoroughly anonymized and aggregated before results are shared publicly with the community. The included contact information will only be used for purposes relating to the surveys, not for any other foundation activities.
Please let me know if you have any questions, of course! -- Jeremy Stanley