[heat][mistral][tc] Status of ply
Hi, As you might know, some of the OpenStack projects, heat and mistral, use yaql to support flexible user input. The yaql library has been tiny and stable, thanks to the ply[1] library which is the python-written implementation of lex and yacc . However, it was declared that no further release of ply is planned[2] and the author now suggests copying it into the project[3]. The old released version has been working so far, but I'm unsure if it could survive through recent drastic changes in python package tooling. Looking at the code, it seems all codes are tagged with BSD 3-clause license[3] though the repository lacks the LICENSE file. I guess that the easiest approach is to import the codes to somewhere within yaql (like yaql._ply) and use the imported codes, with a note explaining these codes are imported from ply. This follows the suggestion by the author, but I'd like to get some guidance from TC to make sure it won't break the whole license expectation. An alternative approach is to implement our own parser, though I'm not too sure if that's feasible at least from my PoV. If we want this approach, then we need actual volunteer for it who will immediately start working on it. If neither of the two options above work, we might end up removing yaql functionality completely from OpenStack. AFAIK the one in heat was added for TripleO which was already retired so the impact may be limited, while for mistral removal could have lager impact. Thank you, Takashi [1] https://github.com/dabeaz/ply [2] https://github.com/dabeaz/ply?tab=readme-ov-file#important-notice---october-... [3] https://github.com/dabeaz/ply?tab=readme-ov-file#questions-and-answers [4] https://github.com/dabeaz/ply/blob/master/src/ply/lex.py#L10-L33 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
On Sat, Sep 13, 2025 at 10:23 PM Takashi Kajinami <kajinamit@oss.nttdata.com> wrote:
Hi,
As you might know, some of the OpenStack projects, heat and mistral, use yaql to support flexible user input. The yaql library has been tiny and stable, thanks to the ply[1] library which is the python-written implementation of lex and yacc .
However, it was declared that no further release of ply is planned[2] and the author now suggests copying it into the project[3]. The old released version has been working so far, but I'm unsure if it could survive through recent drastic changes in python package tooling.
Looking at the code, it seems all codes are tagged with BSD 3-clause license[3] though the repository lacks the LICENSE file. I guess that the easiest approach is to import the codes to somewhere within yaql (like yaql._ply) and use the imported codes, with a note explaining these codes are imported from ply. This follows the suggestion by the author, but I'd like to get some guidance from TC to make sure it won't break the whole license expectation.
+1 Thanks for raising this, and for the due diligence in checking about the licensing and the maintainer's wishes. We discussed this during the TC meeting on September 16th 2025 [5] There's a TC resolution that specifies licensing requirements for projects under OpenStack governance [6]. We don't have an official vendoring policy. As long as you adhere to retaining the original licenses in the source files, using BSD-3 licensed code in OpenStack projects for this purpose seems okay. I like that you're planning to import this as a private module under openstack/yaql so heat and mistral can work without code changes.
An alternative approach is to implement our own parser, though I'm not too sure if that's feasible at least from my PoV. If we want this approach, then we need actual volunteer for it who will immediately start working on it.
If neither of the two options above work, we might end up removing yaql functionality completely from OpenStack. AFAIK the one in heat was added for TripleO which was already retired so the impact may be limited, while for mistral removal could have lager impact.
Thank you, Takashi
[1] https://github.com/dabeaz/ply [2] https://github.com/dabeaz/ply?tab=readme-ov-file#important-notice---october-... [3] https://github.com/dabeaz/ply?tab=readme-ov-file#questions-and-answers [4] https://github.com/dabeaz/ply/blob/master/src/ply/lex.py#L10-L33
[5] https://meetings.opendev.org/meetings/tc/2025/tc.2025-09-16-17.00.log.html#l... [6] https://governance.openstack.org/tc/reference/licensing.html
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
Hi, Thank you Takashi for bringing that to mind. YAQL is on the key piece of mistral and there is no plan to get rid of it for now, so I vote for the option to vendorize ply inside the yaql lib if possible. Do I understand correctly that you will try to achieve this? Let me know if I can help you in that topic. Cheers, Arnaud On 16.09.25 - 11:45, Goutham Pacha Ravi wrote:
On Sat, Sep 13, 2025 at 10:23 PM Takashi Kajinami <kajinamit@oss.nttdata.com> wrote:
Hi,
As you might know, some of the OpenStack projects, heat and mistral, use yaql to support flexible user input. The yaql library has been tiny and stable, thanks to the ply[1] library which is the python-written implementation of lex and yacc .
However, it was declared that no further release of ply is planned[2] and the author now suggests copying it into the project[3]. The old released version has been working so far, but I'm unsure if it could survive through recent drastic changes in python package tooling.
Looking at the code, it seems all codes are tagged with BSD 3-clause license[3] though the repository lacks the LICENSE file. I guess that the easiest approach is to import the codes to somewhere within yaql (like yaql._ply) and use the imported codes, with a note explaining these codes are imported from ply. This follows the suggestion by the author, but I'd like to get some guidance from TC to make sure it won't break the whole license expectation.
+1 Thanks for raising this, and for the due diligence in checking about the licensing and the maintainer's wishes. We discussed this during the TC meeting on September 16th 2025 [5]
There's a TC resolution that specifies licensing requirements for projects under OpenStack governance [6]. We don't have an official vendoring policy. As long as you adhere to retaining the original licenses in the source files, using BSD-3 licensed code in OpenStack projects for this purpose seems okay. I like that you're planning to import this as a private module under openstack/yaql so heat and mistral can work without code changes.
An alternative approach is to implement our own parser, though I'm not too sure if that's feasible at least from my PoV. If we want this approach, then we need actual volunteer for it who will immediately start working on it.
If neither of the two options above work, we might end up removing yaql functionality completely from OpenStack. AFAIK the one in heat was added for TripleO which was already retired so the impact may be limited, while for mistral removal could have lager impact.
Thank you, Takashi
[1] https://github.com/dabeaz/ply [2] https://github.com/dabeaz/ply?tab=readme-ov-file#important-notice---october-... [3] https://github.com/dabeaz/ply?tab=readme-ov-file#questions-and-answers [4] https://github.com/dabeaz/ply/blob/master/src/ply/lex.py#L10-L33
[5] https://meetings.opendev.org/meetings/tc/2025/tc.2025-09-16-17.00.log.html#l... [6] https://governance.openstack.org/tc/reference/licensing.html
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
Yes I already prepared https://review.opendev.org/c/openstack/yaql/+/961398 , but would be nice if I can ask someone else to review it. On 9/18/25 4:53 PM, Arnaud Morin wrote:
Hi,
Thank you Takashi for bringing that to mind. YAQL is on the key piece of mistral and there is no plan to get rid of it for now, so I vote for the option to vendorize ply inside the yaql lib if possible.
Do I understand correctly that you will try to achieve this? Let me know if I can help you in that topic.
Cheers, Arnaud
On 16.09.25 - 11:45, Goutham Pacha Ravi wrote:
On Sat, Sep 13, 2025 at 10:23 PM Takashi Kajinami <kajinamit@oss.nttdata.com> wrote:
Hi,
As you might know, some of the OpenStack projects, heat and mistral, use yaql to support flexible user input. The yaql library has been tiny and stable, thanks to the ply[1] library which is the python-written implementation of lex and yacc .
However, it was declared that no further release of ply is planned[2] and the author now suggests copying it into the project[3]. The old released version has been working so far, but I'm unsure if it could survive through recent drastic changes in python package tooling.
Looking at the code, it seems all codes are tagged with BSD 3-clause license[3] though the repository lacks the LICENSE file. I guess that the easiest approach is to import the codes to somewhere within yaql (like yaql._ply) and use the imported codes, with a note explaining these codes are imported from ply. This follows the suggestion by the author, but I'd like to get some guidance from TC to make sure it won't break the whole license expectation.
+1 Thanks for raising this, and for the due diligence in checking about the licensing and the maintainer's wishes. We discussed this during the TC meeting on September 16th 2025 [5]
There's a TC resolution that specifies licensing requirements for projects under OpenStack governance [6]. We don't have an official vendoring policy. As long as you adhere to retaining the original licenses in the source files, using BSD-3 licensed code in OpenStack projects for this purpose seems okay. I like that you're planning to import this as a private module under openstack/yaql so heat and mistral can work without code changes.
An alternative approach is to implement our own parser, though I'm not too sure if that's feasible at least from my PoV. If we want this approach, then we need actual volunteer for it who will immediately start working on it.
If neither of the two options above work, we might end up removing yaql functionality completely from OpenStack. AFAIK the one in heat was added for TripleO which was already retired so the impact may be limited, while for mistral removal could have lager impact.
Thank you, Takashi
[1] https://github.com/dabeaz/ply [2] https://github.com/dabeaz/ply?tab=readme-ov-file#important-notice---october-... [3] https://github.com/dabeaz/ply?tab=readme-ov-file#questions-and-answers [4] https://github.com/dabeaz/ply/blob/master/src/ply/lex.py#L10-L33
[5] https://meetings.opendev.org/meetings/tc/2025/tc.2025-09-16-17.00.log.html#l... [6] https://governance.openstack.org/tc/reference/licensing.html
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit
On 9/18/25 12:21 PM, Takashi Kajinami wrote:
Yes I already prepared https://review.opendev.org/c/openstack/yaql/ +/961398 , but would be nice if I can ask someone else to review it.
In what way that's better than taking over the lib itself, and maintaining it? Vendoring is bad... The fact that upstream for ply says we should do that doesn't change the fact. Cheers, Thomas Goirand (zigo)
On 9/19/25 6:17 AM, Thomas Goirand wrote:
On 9/18/25 12:21 PM, Takashi Kajinami wrote:
Yes I already prepared https://review.opendev.org/c/openstack/yaql/ +/961398 , but would be nice if I can ask someone else to review it.
In what way that's better than taking over the lib itself, and maintaining it?
If you read the README in the ply repository, you may find that the author is willing to keep the repository under his control while he intentionally stopped new release to avoid himself from being part of "your supply chain" so it does not sound feasible to take over the repo.
Vendoring is bad... The fact that upstream for ply says we should do that doesn't change the fact.
It may sound bad to you but that's what the author suggests. I'd avoid forking the project because we are only interested in our own usage within yaql. Given how widely yaql may be used I'm unsure if we can maintain it as a full fork project, unless someone actually steps up here to maintain the full fork.
Cheers,
Thomas Goirand (zigo)
participants (4)
-
Arnaud Morin
-
Goutham Pacha Ravi
-
Takashi Kajinami
-
Thomas Goirand