Hi All, Is there any discussion going on among the developers or architects of this community to change the coding language from Python to Go in the near future by any chance ? Or, are there any such suggestions from the community so far, given the better speed of Go over Python in execution times ? Thanks Y.G,
Oh really, first time I heard this. Is it that easy to just convert python to go ? Agreed on speed though we have switch many applications from py to go just because of multi threading requirement and speeeeeed Sent from my iPhone
On Sep 19, 2023, at 8:11 AM, Gk Gk <ygk.kmr@gmail.com> wrote:
Hi All,
Is there any discussion going on among the developers or architects of this community to change the coding language from Python to Go in the near future by any chance ? Or, are there any such suggestions from the community so far, given the better speed of Go over Python in execution times ?
Thanks Y.G,
I am just asking the community if there are any such plans for the future. I am not saying that it is decided on . On Tue, Sep 19, 2023 at 5:45 PM Satish Patel <satish.txt@gmail.com> wrote:
Oh really, first time I heard this. Is it that easy to just convert python to go ?
Agreed on speed though we have switch many applications from py to go just because of multi threading requirement and speeeeeed
Sent from my iPhone
On Sep 19, 2023, at 8:11 AM, Gk Gk <ygk.kmr@gmail.com> wrote:
Hi All,
Is there any discussion going on among the developers or architects of this community to change the coding language from Python to Go in the near future by any chance ? Or, are there any such suggestions from the community so far, given the better speed of Go over Python in execution times ?
Thanks Y.G,
On 2023-09-19 17:38:42 +0530 (+0530), Gk Gk wrote:
Is there any discussion going on among the developers or architects of this community to change the coding language from Python to Go in the near future by any chance ? Or, are there any such suggestions from the community so far, given the better speed of Go over Python in execution times ?
Not really, no. Completely rewriting software in a different language is a massive undertaking, and when you consider it's taken 13 years to build the software to its current state that's a lot to rewrite from scratch. There was, at one point, an attempt to replace a performance-critical subservice of Swift with something written in Go (called Hummingbird), and while that plan seemed to hold promise the work on integrating it eventually dried up. Perhaps someone from that team can speak more to the reasons it didn't take off (pun intended). Keep in mind that most of OpenStack's own software is often not in performance-critical paths, it frequently calls out to other programs in userspace that are already written in "faster" languages, so the relative slowness of the Python parts of that system doesn't necessarily create much additional overhead anyway, therefore the benefits of rewriting them might not always be as great as you anticipate. It comes down to a cost vs. benefit ratio, where the cost of rewriting is very high and the benefits aren't, or can't clearly be proven as, good enough to outweigh those costs. Python (well, the popular CPython interpreter anyway) is itself getting faster too, it's just not been a high priority for their community until the last few years, but more recently they've invested a lot of effort in making efficiency and performance improvements. Take a look at the speed improvements which came out of the Faster CPython effort in 3.10 and 3.11 as well as the 3.12 release candidates for example, or the free-threading/no-GIL work for PEP 703. Why rewrite everything into a faster language when you could instead just make the language it's already written in faster instead? -- Jeremy Stanley
Am Dienstag, dem 19.09.2023 um 12:31 +0000 schrieb Jeremy Stanley:
Not really, no. Completely rewriting software in a different language is a massive undertaking, and when you consider it's taken 13 years to build the software to its current state that's a lot to rewrite from scratch.
agreed
Keep in mind that most of OpenStack's own software is often not in performance-critical paths, it frequently calls out to other programs in userspace that are already written in "faster" languages, so the relative slowness of the Python parts of that system doesn't necessarily create much additional overhead anyway, therefore the benefits of rewriting them might not always be as great as you anticipate. It comes down to a cost vs. benefit ratio, where the cost of rewriting is very high and the benefits aren't, or can't clearly be proven as, good enough to outweigh those costs.
Python (well, the popular CPython interpreter anyway) is itself getting faster too, it's just not been a high priority for their community until the last few years, but more recently they've invested a lot of effort in making efficiency and performance improvements. Take a look at the speed improvements which came out of the Faster CPython effort in 3.10 and 3.11 as well as the 3.12 release candidates for example, or the free-threading/no-GIL work for PEP 703. Why rewrite everything into a faster language when you could instead just make the language it's already written in faster instead?
well, if you have ever run any moderate sized openstack cluster you must disagree here. of course many openstack components *are* performance sensitive, especially if you run k8s on top of it which can hammer the api servers with thousands requests per second. I don't even want to go into arguments about general efficiency of computing, preservation of resources and "green it" [0]. a good rewrite would possibly use rust instead of golang (there, I said it), because it's easier to integrate with python[1] as it has a better FFI story than golang, from what I read online. This way this could be done in an incremental way. This would imho be the only feasible approach, which could also yield new developers interested in openstack/cloud computing. e.g. Artem rewrote the current openstack cli client in rust and while the current cli in python is really slow for larger clouds, it is fast in rust[2]. So there is some work being done, although it's very very early days. the results of the poc where shown at the last public cloud sig meeting, which was quite interesting. But I do agree, as said above, that it's hard to do, and there seems sadly to be not much buy-in from the community at large just yet. [0]: https://thenewstack.io/which-programming-languages-use-the-least-electricity... [1]: https://pyo3.rs [2]: https://lists.openstack.org/pipermail/openstack-discuss/2023-August/034934.h... kind regards -- Sven Kieske Senior Cloud Engineer Mail: kieske@osism.tech Web: https://osism.tech OSISM GmbH Teckstraße 62 / 70190 Stuttgart / Deutschland Geschäftsführer: Christian Berendt Unternehmenssitz: Stuttgart Amtsgericht: Stuttgart, HRB 756139
On 2023-09-19 17:55:07 +0200 (+0200), Sven Kieske wrote: [...]
e.g. Artem rewrote the current openstack cli client in rust and while the current cli in python is really slow for larger clouds, it is fast in rust[2]. So there is some work being done, although it's very very early days.
the results of the poc where shown at the last public cloud sig meeting, which was quite interesting. [...]
It's worth keeping in mind that any rewrite, even into the same language, is likely to yield performance improvements. When redoing an application from scratch you have the opportunity to make different architectural/design decisions, free yourself from cruft and historical baggage, and so on. It's hard to say "OpenStackCLI rewritten in Rust is faster because Rust is faster than Python" unless the design is architecturally the same (which is also almost impossible to achieve between different languages anyway). Most of the current CLI's startup slowness is because of the mechanism it relies on for finding plugins, and even just redoing that with a more performant alternative would make a huge difference. Regardless, that's a cool feat, and like I said focusing on bits in performance-critical paths which can take advantage of specific language features makes more sense than blindly rewriting everything we've got in another language. But also Python is getting better. It would be cool to try comparing DevStack/Tempest job durations and resource consumption between 3.8 and 3.11 for example (or 3.12 once we can). -- Jeremy Stanley
Hi, On 9/19/23 14:08, Gk Gk wrote:
Hi All,
Is there any discussion going on among the developers or architects of this community to change the coding language from Python to Go in the near future by any chance ? Or, are there any such suggestions from the community so far, given the better speed of Go over Python in execution times ?
If we're after speed, we need to look at Rust instead (and let the flame war begin!) On a serious note, I don't believe any increment in speed will realistically justify the many years (!) of work to replace the existing code base exactly. And many projects won't see much increment of speed at all - namely, all projects that are mostly bound by external communications. Take Ironic. It may take well over 10 seconds to issue a power on request via IPMI. Any win we can make in the surrounding glue code is simply negligible compared to that. I know that Swift was looking at Go because of their specific I/O operation requirements, but that's probably it. At the same time, OpenStack simply don't have enough people to make that happen. Nor can we expect the participating companies to spend their money on that. Dmitry P.S. Just today, I could not compile a medium-sized Go project because my laptop has "only" 12 GiB of RAM, so the OOM killer kicks in. Talk about efficiency and performance...
Thanks Y.G,
participants (5)
-
Dmitry Tantsur
-
Gk Gk
-
Jeremy Stanley
-
Satish Patel
-
Sven Kieske