Concluding the AI Policy revision?
(Kicking off the discussion here, since the June board meeting was cancelled) Colleagues, It turns out the proposed "Assisted-By" commit message label is already in use, as can be seen from these two in-flight reviews: nova-specs/+/951222[1] owner: Sean Mooney Assisted-By: deepseek-r1 Assisted-By: google-gemini-flash-2.5 project-team-guide/+/948484[2] owner: Goutham Pacha Ravi Assisted-By: OpenAI ChatGPT (As an aside: there's nice symmetry to the nova-specs one, as I first heard the new "Assisted-By" label suggested by Sean back in early April, in advance of the board discussion) Also worth noting the actual choice of models in this small sample suggests not much weight is being put on the current policy's steer towards: "exclusively using compatible licensed inputs to train the model, using a model released as open source, using a model trained exclusively on compatible licensed code". To catch up with emerging practice, I'd suggest that we augment the formal policy with "Assisted-By" as soon as possible. FWIW, I think we should also simplify, shorten, and make the policy more permissive overall (e.g. by removing the language on the choice of model, and pushing the secondary scanning into the code review or merge gating flow). Though at least we should get the Assisted-By option codified soon. Thanks, Eoghan [1] https://review.opendev.org/c/openstack/nova-specs/+/951222 [2] https://review.opendev.org/c/openstack/project-team-guide/+/948484 -- Eoghan Glynn He/Him/His Senior Director, OpenStack Engineering Dublin, Ireland M: +353-86-8175024 <https://www.redhat.com>
On Wed, Jun 11, 2025 at 4:02 AM Eoghan Glynn <eglynn@redhat.com> wrote:
(Kicking off the discussion here, since the June board meeting was cancelled)
Colleagues,
It turns out the proposed "Assisted-By" commit message label is already in use, as can be seen from these two in-flight reviews:
nova-specs/+/951222[1] owner: Sean Mooney Assisted-By: deepseek-r1 Assisted-By: google-gemini-flash-2.5
project-team-guide/+/948484[2] owner: Goutham Pacha Ravi Assisted-By: OpenAI ChatGPT
(As an aside: there's nice symmetry to the nova-specs one, as I first heard the new "Assisted-By" label suggested by Sean back in early April, in advance of the board discussion)
Also worth noting the actual choice of models in this small sample suggests not much weight is being put on the current policy's steer towards: "exclusively using compatible licensed inputs to train the model, using a model released as open source, using a model trained exclusively on compatible licensed code".
Indeed, this can be observed with the Generated-By label. A huge amount of the feedback we've gotten from non-Red Hat folks is that the policy is basically too long and needs to be more like a single paragraph. I have an outstanding todo to revisit the draft revision I started a month or so ago and try to craft a TL;DR section like what was done with the code of conduct, I've just not had time.
To catch up with emerging practice, I'd suggest that we augment the formal policy with "Assisted-By" as soon as possible. FWIW, I think we should also simplify, shorten, and make the policy more permissive overall (e.g. by removing the language on the choice of model, and pushing the secondary scanning into the code review or merge gating flow). Though at least we should get the Assisted-By option codified soon.
I concur we need to regarding Assisted-By, unfortunately the push to not hold a meeting from the staff due to onboarding pulled the wind from my sail a couple weeks ago. The path forward seems to be to finish revisions to a draft, get a quick legal review, and then bring it up as an item to vote on in the next meeting, if we're able to achieve the necessary review steps in time.
Thanks, Eoghan
[1] https://review.opendev.org/c/openstack/nova-specs/+/951222 [2] https://review.opendev.org/c/openstack/project-team-guide/+/948484
--
Eoghan Glynn
He/Him/His
Senior Director, OpenStack Engineering
Dublin, Ireland
M: +353-86-8175024 <https://www.redhat.com> _______________________________________________ Foundation-board mailing list -- foundation-board@lists.openinfra.org To unsubscribe send an email to foundation-board-leave@lists.openinfra.org
On 5/30/25 4:01 PM, Eoghan Glynn wrote:
Also worth noting the actual choice of models in this small sample suggests not much weight is being put on the current policy's steer towards: "exclusively using compatible licensed inputs to train the model, using a model released as open source, using a model trained exclusively on compatible licensed code".
To catch up with emerging practice, I'd suggest that we augment the formal policy with "Assisted-By" as soon as possible. FWIW, I think we should also simplify, shorten, and make the policy more permissive overall (e.g. by removing the language on the choice of model, and pushing the secondary scanning into the code review or merge gating flow). Though at least we should get the Assisted-By option codified soon.
Agreed on simplifications and adding Assisted-By as an option. But, this is not the time to remove the clear reminder to contributors that there is real uncertainty and risk around AI tools and whether their outputs can be used in open source software. Tagging the commits does mitigate some of the risk (it at least gives us the option to rip out that code someday if we have to), but making smart choices about which tools you use in the first place is even better. The world is really only beginning the journey of understanding the legal implications of AI generation in the context of copyrighted works, for example: https://www.wired.com/story/disney-universal-sue-midjourney/ Allison
On Thu, Jun 12, 2025 at 1:11 PM Allison Randal <allison@lohutok.net> wrote:
On 5/30/25 4:01 PM, Eoghan Glynn wrote:
Also worth noting the actual choice of models in this small sample suggests not much weight is being put on the current policy's steer towards: "exclusively using compatible licensed inputs to train the model, using a model released as open source, using a model trained exclusively on compatible licensed code".
To catch up with emerging practice, I'd suggest that we augment the formal policy with "Assisted-By" as soon as possible. FWIW, I think we should also simplify, shorten, and make the policy more permissive overall (e.g. by removing the language on the choice of model, and pushing the secondary scanning into the code review or merge gating flow). Though at least we should get the Assisted-By option codified
soon.
Agreed on simplifications and adding Assisted-By as an option.
But, this is not the time to remove the clear reminder to contributors that there is real uncertainty and risk around AI tools and whether their outputs can be used in open source software. Tagging the commits does mitigate some of the risk (it at least gives us the option to rip out that code someday if we have to), but making smart choices about which tools you use in the first place is even better.
The world is really only beginning the journey of understanding the legal implications of AI generation in the context of copyrighted works, for example:
https://www.wired.com/story/disney-universal-sue-midjourney/
Totally get the concern about the risks of future litigation. We clearly have vanishingly low invocation of the existing policy though, at least for the OpenStack codebases. Now that could be due to one of several reasons, including: (a) very low genAI tool usage among our contributors or: (b) widespread non-observance of the policy's requirements IMO both are bad outcomes for the community, since (a) cuts us off from the productivity gains accruing from using these tools, whereas (b) cuts us off from the future risk mitigation of clearly marked AI-assisted contributions. FWIW I suspect that the current wording of the policy could be a factor in either (a) or (b). and that a simplified, lightweight policy is more likely to be widely understood and observed. Cheers, Eoghan
Allison
On 6/12/25 4:01 PM, Eoghan Glynn wrote:
We clearly have vanishingly low invocation of the existing policy though, at least for the OpenStack codebases.
Now that could be due to one of several reasons, including:
(a) very low genAI tool usage among our contributors
or:
(b) widespread non-observance of the policy's requirements
or: (c) generative AI is good at producing code when it was trained on a million examples of similar code (like using a popular javascript library), but not good at producing code that's rare, unusual, or in a unique problem domain. An LLM trained on Nova code might be useful for writing Nova code, but most generative AI tools won't be. It sounds like a good next step would be to have a conversation with developers, rather than making assumptions about what developers want or need. Allison
On Fri, Jun 13, 2025 at 5:02 PM Allison Randal <allison@lohutok.net> wrote:
On 6/12/25 4:01 PM, Eoghan Glynn wrote:
We clearly have vanishingly low invocation of the existing policy though, at least for the OpenStack codebases.
Now that could be due to one of several reasons, including:
(a) very low genAI tool usage among our contributors
or:
(b) widespread non-observance of the policy's requirements
or:
(c) generative AI is good at producing code when it was trained on a million examples of similar code (like using a popular javascript library), but not good at producing code that's rare, unusual, or in a unique problem domain. An LLM trained on Nova code might be useful for writing Nova code, but most generative AI tools won't be.
Possible, though TBH I'd be surprised if the models in wide use today didn't include in their training corpus most (if not all) public and permissively licensed github repos that meet some basic quality/activity threshold. So I'd suspect the Nova code is likely in the training mix already for the models people are actually using right now ... though of course, open to correction on that.
It sounds like a good next step would be to have a conversation with developers, rather than making assumptions about what developers want or need.
Fair enough, happy to have a wider conversation. Of the small sample I've already talked to, seemed the current policy was seen as a bit too heavyweight. IIUC similar sentiments <https://etherpad.opendev.org/p/board-informal-ai-contribution#L68> came up in the discussion that Julia ran a while back, FWIW. Cheers, Eoghan
Allison
On Fri, Jun 13, 2025 at 10:03 AM Eoghan Glynn <eglynn@redhat.com> wrote:
On Fri, Jun 13, 2025 at 5:02 PM Allison Randal <allison@lohutok.net> wrote:
On 6/12/25 4:01 PM, Eoghan Glynn wrote:
We clearly have vanishingly low invocation of the existing policy though, at least for the OpenStack codebases.
Now that could be due to one of several reasons, including:
(a) very low genAI tool usage among our contributors
or:
(b) widespread non-observance of the policy's requirements
or:
(c) generative AI is good at producing code when it was trained on a million examples of similar code (like using a popular javascript library), but not good at producing code that's rare, unusual, or in a unique problem domain. An LLM trained on Nova code might be useful for writing Nova code, but most generative AI tools won't be.
Possible, though TBH I'd be surprised if the models in wide use today didn't include in their training corpus most (if not all) public and permissively licensed github repos that meet some basic quality/activity threshold.
So I'd suspect the Nova code is likely in the training mix already for the models people are actually using right now ... though of course, open to correction on that.
It sounds like a good next step would be to have a conversation with developers, rather than making assumptions about what developers want or need.
Fair enough, happy to have a wider conversation. Of the small sample I've already talked to, seemed the current policy was seen as a bit too heavyweight. IIUC similar sentiments came up in the discussion that Julia ran a while back, FWIW.
Indeed. This is all as the tools have evolved substantially. We've gone from bad code, to slightly better code. And then even better code, where we can ask it to rewrite the suggestion with OpenStack context and even context of a project's code base. And now, we're at "productivity tool" level of capability as some tools are able to understand and carry out specific user-requested actions which are an actual productivity boost as opposed to just "write" code. Two recent examples I'm aware of: - "find my bug in this code, I'm experiencing it hang this way" which successfully led to some solutions to explore. - "Please restructure the code base to change the folder of this module and update all references", similar to running "sed s/cmd/command/g" across the code base and "mv cmd command". It didn't get everything right, but it was surprisingly close. So I think the right thing is to shift our focus to expect that to be the use case, and not the bulk composition of code which we originally envisioned early on. I don't believe we should greatly dial back the policy document, but we do need to focus our engagement because the number one piece of feedback which continues to resonate to me is that it was just too much for folks to wrap their heads around. We wrote it from the context of the board assuming a similarly leveled reader. And... turns out newer contributors are the folks reading it and their eyes glaze over around the third paragraph. While we should absolutely continue to discuss, we do have a need to revise the document based upon existing input and attempt to focus it. My plan for tomorrow morning, unless something is on fire first thing, is to send out my current working draft with comment permission enabled to enable us to further collaborate and move this item forward. Furthermore, I think it would be good to schedule a follow-up call on this topic, perhaps next Tuesday? Thanks, -Julia
Cheers, Eoghan
Allison
_______________________________________________ Foundation-board mailing list -- foundation-board@lists.openinfra.org To unsubscribe send an email to foundation-board-leave@lists.openinfra.org
On Tue, Jun 17, 2025 at 11:36 PM Julia Kreger <juliaashleykreger@gmail.com> wrote:
On Fri, Jun 13, 2025 at 10:03 AM Eoghan Glynn <eglynn@redhat.com> wrote:
On Fri, Jun 13, 2025 at 5:02 PM Allison Randal <allison@lohutok.net>
On 6/12/25 4:01 PM, Eoghan Glynn wrote:
We clearly have vanishingly low invocation of the existing policy though, at least for the OpenStack codebases.
Now that could be due to one of several reasons, including:
(a) very low genAI tool usage among our contributors
or:
(b) widespread non-observance of the policy's requirements
or:
(c) generative AI is good at producing code when it was trained on a million examples of similar code (like using a popular javascript library), but not good at producing code that's rare, unusual, or in a unique problem domain. An LLM trained on Nova code might be useful for writing Nova code, but most generative AI tools won't be.
Possible, though TBH I'd be surprised if the models in wide use today didn't include in their training corpus most (if not all) public and
wrote: permissively licensed github repos that meet some basic quality/activity threshold.
So I'd suspect the Nova code is likely in the training mix already for
the models people are actually using right now ... though of course, open to correction on that.
It sounds like a good next step would be to have a conversation with developers, rather than making assumptions about what developers want or need.
Fair enough, happy to have a wider conversation. Of the small sample
I've already talked to, seemed the current policy was seen as a bit too heavyweight. IIUC similar sentiments came up in the discussion that Julia ran a while back, FWIW.
Indeed. This is all as the tools have evolved substantially. We've gone from bad code, to slightly better code. And then even better code, where we can ask it to rewrite the suggestion with OpenStack context and even context of a project's code base. And now, we're at "productivity tool" level of capability as some tools are able to understand and carry out specific user-requested actions which are an actual productivity boost as opposed to just "write" code.
Two recent examples I'm aware of:
- "find my bug in this code, I'm experiencing it hang this way" which successfully led to some solutions to explore. - "Please restructure the code base to change the folder of this module and update all references", similar to running "sed s/cmd/command/g" across the code base and "mv cmd command". It didn't get everything right, but it was surprisingly close.
So I think the right thing is to shift our focus to expect that to be the use case, and not the bulk composition of code which we originally envisioned early on.
I don't believe we should greatly dial back the policy document, but we do need to focus our engagement because the number one piece of feedback which continues to resonate to me is that it was just too much for folks to wrap their heads around. We wrote it from the context of the board assuming a similarly leveled reader. And... turns out newer contributors are the folks reading it and their eyes glaze over around the third paragraph.
Thanks Julia, If there isn't an appetite to dial back the main policy doc, then one obvious approach would be to also provide a super-concise & informal preamble, deliberately written in very approachable language that'll avoid the eye-glaze. Something along of the lines of: "We welcome contributions crafted with the help of AI tools. Use whatever model works best for you, just check the terms and conditions first. You'll need to mark patches prepared with the help of AI. Use either the 'Generated-by' label when you got lots of help, or else 'Assisted-by' when you still wrote most of it by hand. A human must always be in the loop, both to submit and review patches. Be extra careful with security fixes. Copyright law still applies to any preexisting work. And of course, follow your employer’s rules where relevant." (not wedded to that particular formula of words, just intended as an illustration of the type of thing that might work)
While we should absolutely continue to discuss, we do have a need to revise the document based upon existing input and attempt to focus it. My plan for tomorrow morning, unless something is on fire first thing, is to send out my current working draft with comment permission enabled to enable us to further collaborate and move this item forward.
Furthermore, I think it would be good to schedule a follow-up call on this topic, perhaps next Tuesday?
Great idea! Cheers, Eoghan
Thanks,
-Julia
Cheers, Eoghan
Allison
_______________________________________________ Foundation-board mailing list -- foundation-board@lists.openinfra.org To unsubscribe send an email to foundation-board-leave@lists.openinfra.org
On 6/18/25 4:35 AM, Eoghan Glynn wrote:
On Tue, Jun 17, 2025 at 11:36 PM Julia Kreger <juliaashleykreger@gmail.com <mailto:juliaashleykreger@gmail.com>> wrote:
Indeed. This is all as the tools have evolved substantially. We've gone from bad code, to slightly better code. And then even better code, where we can ask it to rewrite the suggestion with OpenStack context and even context of a project's code base. And now, we're at "productivity tool" level of capability as some tools are able to understand and carry out specific user-requested actions which are an actual productivity boost as opposed to just "write" code.
Two recent examples I'm aware of:
- "find my bug in this code, I'm experiencing it hang this way" which successfully led to some solutions to explore. - "Please restructure the code base to change the folder of this module and update all references", similar to running "sed s/cmd/command/g" across the code base and "mv cmd command". It didn't get everything right, but it was surprisingly close.
So I think the right thing is to shift our focus to expect that to be the use case, and not the bulk composition of code which we originally envisioned early on.
Nod, this makes sense. An AI tool to improve Nova code that a human wrote really only requires training on Python code, and there are loads of public examples of that. It's completely different than an AI tool generating working Nova code from a prompt/comment. (Remember, generative AI produces statistically likely output, so it's really only good at generating things it has seen many, many examples of.)
I don't believe we should greatly dial back the policy document, but we do need to focus our engagement because the number one piece of feedback which continues to resonate to me is that it was just too much for folks to wrap their heads around. We wrote it from the context of the board assuming a similarly leveled reader. And... turns out newer contributors are the folks reading it and their eyes glaze over around the third paragraph.
Thanks Julia,
If there isn't an appetite to dial back the main policy doc, then one obvious approach would be to also provide a super-concise & informal preamble, deliberately written in very approachable language that'll avoid the eye-glaze.
Something along of the lines of:
"We welcome contributions crafted with the help of AI tools. Use whatever model works best for you, just check the terms and conditions first. You'll need to mark patches prepared with the help of AI. Use either the 'Generated-by' label when you got lots of help, or else 'Assisted-by' when you still wrote most of it by hand. A human must always be in the loop, both to submit and review patches. Be extra careful with security fixes. Copyright law still applies to any preexisting work. And of course, follow your employer’s rules where relevant."
(not wedded to that particular formula of words, just intended as an illustration of the type of thing that might work)
I like that friendly intro. I also think we can make the policy doc easier to read (while keeping the original intended meaning), and shift the emphasis slightly to highlight using AI tools for assistance to improve human written code.
While we should absolutely continue to discuss, we do have a need to revise the document based upon existing input and attempt to focus it. My plan for tomorrow morning, unless something is on fire first thing, is to send out my current working draft with comment permission enabled to enable us to further collaborate and move this item forward.
Furthermore, I think it would be good to schedule a follow-up call on this topic, perhaps next Tuesday?
Works for me. Allison
Good morning folks, I've got an early document[0] we can go back and forth on. Currently default access is comment only. If anyone is up for suggesting text, please let me know and I'll gladly grant that access as long as we keep it in suggestion mode so we can easily view the changes. I'll send out a meeting invite for next Tuesday for discussion. Thanks, -Julia [0]: https://docs.google.com/document/d/1SqC0qiBPrEj0ATtHPvSc-PrK5_JhBb_qy5NLZt3O... On Wed, Jun 18, 2025 at 6:55 AM Allison Randal <allison@lohutok.net> wrote:
On 6/18/25 4:35 AM, Eoghan Glynn wrote:
On Tue, Jun 17, 2025 at 11:36 PM Julia Kreger <juliaashleykreger@gmail.com <mailto:juliaashleykreger@gmail.com>> wrote:
Indeed. This is all as the tools have evolved substantially. We've gone from bad code, to slightly better code. And then even better code, where we can ask it to rewrite the suggestion with OpenStack context and even context of a project's code base. And now, we're at "productivity tool" level of capability as some tools are able to understand and carry out specific user-requested actions which are an actual productivity boost as opposed to just "write" code.
Two recent examples I'm aware of:
- "find my bug in this code, I'm experiencing it hang this way" which successfully led to some solutions to explore. - "Please restructure the code base to change the folder of this module and update all references", similar to running "sed s/cmd/command/g" across the code base and "mv cmd command". It didn't get everything right, but it was surprisingly close.
So I think the right thing is to shift our focus to expect that to be the use case, and not the bulk composition of code which we originally envisioned early on.
Nod, this makes sense. An AI tool to improve Nova code that a human wrote really only requires training on Python code, and there are loads of public examples of that. It's completely different than an AI tool generating working Nova code from a prompt/comment.
(Remember, generative AI produces statistically likely output, so it's really only good at generating things it has seen many, many examples of.)
I don't believe we should greatly dial back the policy document, but we do need to focus our engagement because the number one piece of feedback which continues to resonate to me is that it was just too much for folks to wrap their heads around. We wrote it from the context of the board assuming a similarly leveled reader. And... turns out newer contributors are the folks reading it and their eyes glaze over around the third paragraph.
Thanks Julia,
If there isn't an appetite to dial back the main policy doc, then one obvious approach would be to also provide a super-concise & informal preamble, deliberately written in very approachable language that'll avoid the eye-glaze.
Something along of the lines of:
"We welcome contributions crafted with the help of AI tools. Use whatever model works best for you, just check the terms and conditions first. You'll need to mark patches prepared with the help of AI. Use either the 'Generated-by' label when you got lots of help, or else 'Assisted-by' when you still wrote most of it by hand. A human must always be in the loop, both to submit and review patches. Be extra careful with security fixes. Copyright law still applies to any preexisting work. And of course, follow your employer’s rules where relevant."
(not wedded to that particular formula of words, just intended as an illustration of the type of thing that might work)
I like that friendly intro.
I also think we can make the policy doc easier to read (while keeping the original intended meaning), and shift the emphasis slightly to highlight using AI tools for assistance to improve human written code.
While we should absolutely continue to discuss, we do have a need to revise the document based upon existing input and attempt to focus it. My plan for tomorrow morning, unless something is on fire first thing, is to send out my current working draft with comment permission enabled to enable us to further collaborate and move this item forward.
Furthermore, I think it would be good to schedule a follow-up call on this topic, perhaps next Tuesday?
Works for me.
Allison
participants (3)
-
Allison Randal
-
Eoghan Glynn
-
Julia Kreger