Hi All, How many more releases are you planning to include eventlet in them ? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives ? Also, can someone write a small blog on how to use backdoor to debug the nova processes. That would be a great help until then.... Thanks elinux
On Wed, 2024-08-14 at 22:38 +0530, engineer2024 wrote:
Hi All,
How many more releases are you planning to include eventlet in them ? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives ? Also, can someone write a small blog on how to use backdoor to debug the nova processes. That would be a great help until then....
it will be a multi year effort to remove eventlet its a massive change we are starting to work on that but not anytime soon you have asked how to debug without actully explciing your issue a few times perhaps you could acutlly explian what your trying to debug and we could point you in the probable right direciton in general we do not diganose issues by debuging nova or opentsack we try ot ensure there is enough loggin at debug level to be able to deduce what happend form the logs for use that have to supprot downstream distiobutions we do not get the opertunity to log into custoemer systems and use a debugger in porduciton to figure out what when wrong. we have to figure that out form the debug logs and inspecting the code instead. there are way to make nova work in a debguger without using the eventlet backdoor to a limited degree but its not super relyable and we generally dont do it.
Thanks elinux
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here: https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html -- Jeremy Stanley
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc. Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information. But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the python way since pdb supports python prompt as well. Of course I do all this on dev systems though... On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
On Thu, 2024-08-15 at 01:49 +0530, engineer2024 wrote:
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc.
Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information.
But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the python way since pdb supports python prompt as well. Of course I do all this on dev systems though...
this is going to be a long responce and its a bit rambely but im goign to provide you with the context and poitn er try and figure out how to do this for your self. first thing to note is while its possibel it not trivial and the payoff is proably not worth it in the long run. we do not have up to day docs for how to do this because this is not how peole learn, contibute to ro develop nova in genral. nova does have a remote debug facilitate that is sepreate for the eventlet backdor to enable you to use an ide to debug remotely. that was added around 2012 and has worked on an off to vering degrees since then. the first thing to understand about nova and several other project is that they are a distibuted system i.e. the applciaiton that compise a given service like nova run in multiple service which are interconnected vai a message bus with in a service and rest apis are exposed for inter service comunication. when you enable a debugger and stop on a breakpoitn only one of the process is stop and the other contiue to exsculte which means that RPC calls, heathbeats and other asynconos tasks can and will timeout and fail. you cannot debug a distibuted system in teh same way as you woudl a simple applction where all state is executed in a signle main loop. the act of observing the internal behavior of the nova api or nova-comptue agent will change its behavior. nova has suppoort for remote debugging that was added about 12 years ago https://wiki.openstack.org/wiki/Nova/RemoteDebugging https://github.com/openstack/nova/blob/master/nova/debugger.py with that supprot you can use an idea like pycharm to connect to the remote debugger and attempt to debug a process. im currently propsoing removing that as part of the removal of eventlet partly because it shoudl not be needed when we dont have enventlet and mainly because it didnt work very well when i last used it. some ides have supprot fo gevent gevent is a fork of eventlet which was forked form some of hte libiares in stackless python eventlet converts a single thread python interperater in a cooperative concurent multi userland process. what does that mean , each python function is not operating in a seperate os stackframe, it has heap allcoated stack frames that allow context swtichs between userland thread at any tiem a funnction woudl do blocking io the reason single step debugging does not work well with gdb is that when you single step the eventlet engine is often going to end up context switching into the midel of another function. to work around this you just set breakpoint at the line you want to go to next and use continue instead fo single step one of the rules when using eventlet is that if your going to monkey patch you need to monkey patch everyting early in addtion too the remote debuging facilites we can disabl mokey patching entirly. you can do that by settign OS_NOVA_DISABLE_EVENTLET_PATCHING=1 https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L87 this will entirly break nova-compute and several other nova process as they have effectivly infinity loops to pool for rpc messags, monitor hyperviors ectra. if the remote debugger is used it disables patching the thread lib https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L46 that is prartly why usign the remote debugger is kind fo buggy in itsself. the code is not inteded to work that way. these fucntionlatiy mainly exist to allow debuging tests not the actulaly running code. if i need to debug test i normaly jsut comment out https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L83-L89 and run the test in my ides(pycharm,vscodes) test debbuger. i normally code in nano/emacs but is do use ides if i really cant figure out what happening however i genreally start with just adding log lines. when i first started workign on opentack in 2013 i very much liked using an ide and debugger and i woudl often spend time trying to get nova to work in a debugger. over tiem i generally found that that was not worth the effort. keepign a devstack deployment working that way was often more work then doing print staement or better writing functional or unit test to repoduce the problem. you previous mails ask about debuging nova's api. if you want to do that its one of the easiest thigns to debug. first do _not_ use the wsgi script adding apache of modwsgi is just going to make your life harder. just disable eventlet monkey patching, by commetiing or usign the env varible and run the nova-api console script directly under pdb or your debugger of choice. that will allow you to basiclly single step debug without any issues as it will not be using eventlet anymore and it has not infinitly loops or perodic tasks so it mostly just works when not using eventlet. that becase it didnt use eventlet at all in the api until about 2016 and even today it only really uses it in one place in my eventlet removal serise i split nova in to the part that use eventlet directly and the parts that dont in the second path https://review.opendev.org/c/openstack/nova/+/904424 anything under nova/cmd/eventlet/ needs eventlet today everything under nova/cmd/standalone/ does not. the approch of doign an api call and following it in a debugger is not a good way in my expericne to learn how somethign like nova really works. it can hellp to a limtied degree but you cant eaislly debug across multipel processes. a minium nova deployment has 4 process typically more. you can try but each openstack service (nova, neutorn, cinder, ...) is typically compised of several micocservices some like keystone and palcement are just rest apis in front of a db and dont use eventlet at all so are trivial to debug as a result. the rest have non tirival runtime interactions that cant be as eaislly decompsed. becuase of that nova and other project invest in our functional tests to allwo ues to emulate that multi process env and better reason about the code. the debugging functionality is documetned in the contibutor guide https://github.com/openstack/nova/blob/master/doc/source/contributor/develop... but we dont really adverstise that because none of the core maintainers use that any more and it has not been maintained for the better part of a decade. since we cant use this type of debugging with our ci system if we cant debug why a failure happens form the ci logs we add more logs because thyat will be useful for future us and operators in production. so our focus is on logging/monitoring for production rather then for developer eases simply because of the large techdebt that eventlet creates in this aready. i know nuetorn has had some better look with using debuggers then nova has, keystone/placmnet are eventlest free adn are just python web apps/rest apis so should "just work" in any python debugger. nova is prehaps the hardest to debug because of the legacy of the proejct so its not the best palce to start looking how this works. too be clear i have ran nova api, nova-conductor, nova scheduler and nova-compute all in 4 ide windows with 4 debuggers at once it was proably 3 year of working on openstack before i understood how to hack that ot work and since i learned how to properly debug without a debuggger, logs, code inspection, unit/fucntioal testing i have nver really need to do that again. gaining the ablity to run openstack in a debugger simply again, in a maintainable way is deffienly one of the reason im looking forward to removing eventlet but its still a lot of work. im not sure how helpful this was but this is proably as clsoe to a blog post as you will get.
On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
On Wed, 2024-08-14 at 23:06 +0100, smooney@redhat.com wrote:
On Thu, 2024-08-15 at 01:49 +0530, engineer2024 wrote:
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc.
Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information.
But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the python way since pdb supports python prompt as well. Of course I do all this on dev systems though...
this is going to be a long responce and its a bit rambely but im goign to provide you with the context and poitn er try and figure out how to do this for your self.
first thing to note is while its possibel it not trivial and the payoff is proably not worth it in the long run. we do not have up to day docs for how to do this because this is not how peole learn, contibute to ro develop nova in genral.
nova does have a remote debug facilitate that is sepreate for the eventlet backdor to enable you to use an ide to debug remotely. that was added around 2012 and has worked on an off to vering degrees since then.
the first thing to understand about nova and several other project is that they are a distibuted system i.e. the applciaiton that compise a given service like nova run in multiple service which are interconnected vai a message bus with in a service and rest apis are exposed for inter service comunication.
when you enable a debugger and stop on a breakpoitn only one of the process is stop and the other contiue to exsculte which means that RPC calls, heathbeats and other asynconos tasks can and will timeout and fail.
you cannot debug a distibuted system in teh same way as you woudl a simple applction where all state is executed in a signle main loop.
the act of observing the internal behavior of the nova api or nova-comptue agent will change its behavior.
nova has suppoort for remote debugging that was added about 12 years ago https://wiki.openstack.org/wiki/Nova/RemoteDebugging https://github.com/openstack/nova/blob/master/nova/debugger.py
with that supprot you can use an idea like pycharm to connect to the remote debugger and attempt to debug a process.
im currently propsoing removing that as part of the removal of eventlet partly because it shoudl not be needed when we dont have enventlet and mainly because it didnt work very well when i last used it.
some ides have supprot fo gevent
gevent is a fork of eventlet which was forked form some of hte libiares in stackless python eventlet converts a single thread python interperater in a cooperative concurent multi userland process.
by the way i forgot to say if you want to debug eventlet mokey patch code adn your using a debugger that supprot gevent it mostly work with eventlet too if you enabel the gevnet supprot in pycham which is disabeld by defaut. i have had it workk reallly well for months and then not work for years and suddenly start workign agian. so the proablme with usign a debugger with nova is not getting it working initally its keepign it working and setting it up on a new machine if you create a new devstack vm for dev again. its not hard, its hard to do repatbly, so blogs, docs and even automation tends to bit wroth quickly.
what does that mean , each python function is not operating in a seperate os stackframe, it has heap allcoated stack frames that allow context swtichs between userland thread at any tiem a funnction woudl do blocking io
the reason single step debugging does not work well with gdb is that when you single step the eventlet engine is often going to end up context switching into the midel of another function.
to work around this you just set breakpoint at the line you want to go to next and use continue instead fo single step
one of the rules when using eventlet is that if your going to monkey patch you need to monkey patch everyting early
in addtion too the remote debuging facilites we can disabl mokey patching entirly. you can do that by settign OS_NOVA_DISABLE_EVENTLET_PATCHING=1 https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L87
this will entirly break nova-compute and several other nova process as they have effectivly infinity loops to pool for rpc messags, monitor hyperviors ectra.
if the remote debugger is used it disables patching the thread lib https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L46
that is prartly why usign the remote debugger is kind fo buggy in itsself. the code is not inteded to work that way.
these fucntionlatiy mainly exist to allow debuging tests not the actulaly running code.
if i need to debug test i normaly jsut comment out https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L83-L89 and run the test in my ides(pycharm,vscodes) test debbuger. i normally code in nano/emacs but is do use ides if i really cant figure out what happening however i genreally start with just adding log lines.
when i first started workign on opentack in 2013 i very much liked using an ide and debugger and i woudl often spend time trying to get nova to work in a debugger.
over tiem i generally found that that was not worth the effort. keepign a devstack deployment working that way was often more work then doing print staement or better writing functional or unit test to repoduce the problem.
you previous mails ask about debuging nova's api.
if you want to do that its one of the easiest thigns to debug. first do _not_ use the wsgi script adding apache of modwsgi is just going to make your life harder.
just disable eventlet monkey patching, by commetiing or usign the env varible and run the nova-api console script directly under pdb or your debugger of choice. that will allow you to basiclly single step debug without any issues as it will not be using eventlet anymore and it has not infinitly loops or perodic tasks so it mostly just works when not using eventlet.
that becase it didnt use eventlet at all in the api until about 2016 and even today it only really uses it in one place in my eventlet removal serise i split nova in to the part that use eventlet directly and the parts that dont in the second path
https://review.opendev.org/c/openstack/nova/+/904424
anything under nova/cmd/eventlet/ needs eventlet today everything under nova/cmd/standalone/ does not.
the approch of doign an api call and following it in a debugger is not a good way in my expericne to learn how somethign like nova really works. it can hellp to a limtied degree but you cant eaislly debug across multipel processes. a minium nova deployment has 4 process typically more.
you can try but each openstack service (nova, neutorn, cinder, ...) is typically compised of several micocservices some like keystone and palcement are just rest apis in front of a db and dont use eventlet at all so are trivial to debug as a result. the rest have non tirival runtime interactions that cant be as eaislly decompsed. becuase of that nova and other project invest in our functional tests to allwo ues to emulate that multi process env and better reason about the code.
the debugging functionality is documetned in the contibutor guide https://github.com/openstack/nova/blob/master/doc/source/contributor/develop... but we dont really adverstise that because none of the core maintainers use that any more and it has not been maintained for the better part of a decade. since we cant use this type of debugging with our ci system if we cant debug why a failure happens form the ci logs we add more logs because thyat will be useful for future us and operators in production. so our focus is on logging/monitoring for production rather then for developer eases simply because of the large techdebt that eventlet creates in this aready.
i know nuetorn has had some better look with using debuggers then nova has,
keystone/placmnet are eventlest free adn are just python web apps/rest apis so should "just work" in any python debugger.
nova is prehaps the hardest to debug because of the legacy of the proejct so its not the best palce to start looking how this works. too be clear i have ran nova api, nova-conductor, nova scheduler and nova-compute all in 4 ide windows with 4 debuggers at once it was proably 3 year of working on openstack before i understood how to hack that ot work and since i learned how to properly debug without a debuggger, logs, code inspection, unit/fucntioal testing i have nver really need to do that again.
gaining the ablity to run openstack in a debugger simply again, in a maintainable way is deffienly one of the reason im looking forward to removing eventlet but its still a lot of work. im not sure how helpful this was but this is proably as clsoe to a blog post as you will get.
On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
Appreciate taking time to post this. I will look into the options and try them... Have a nice week ahead On Thu, 15 Aug 2024, 03:53 , <smooney@redhat.com> wrote:
On Thu, 2024-08-15 at 01:49 +0530, engineer2024 wrote:
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc.
Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information.
But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the
way since pdb supports python prompt as well. Of course I do all this on dev systems though...
this is going to be a long responce and its a bit rambely but im goign to provide you with the context and poitn er try and figure out how to do this for your self.
first thing to note is while its possibel it not trivial and the payoff is proably not worth it in the long run. we do not have up to day docs for how to do
this is not how peole learn, contibute to ro develop nova in genral.
nova does have a remote debug facilitate that is sepreate for the eventlet backdor to enable you to use an ide to debug remotely. that was added around 2012 and has worked on an off to vering degrees since then.
the first thing to understand about nova and several other project is
i.e. the applciaiton that compise a given service like nova run in multiple service which are interconnected vai a message bus with in a service and rest apis are exposed for inter service comunication.
when you enable a debugger and stop on a breakpoitn only one of the
to exsculte which means that RPC calls, heathbeats and other asynconos tasks can and will timeout and fail.
you cannot debug a distibuted system in teh same way as you woudl a simple applction where all state is executed in a signle main loop.
the act of observing the internal behavior of the nova api or nova-comptue agent will change its behavior.
nova has suppoort for remote debugging that was added about 12 years ago https://wiki.openstack.org/wiki/Nova/RemoteDebugging https://github.com/openstack/nova/blob/master/nova/debugger.py
with that supprot you can use an idea like pycharm to connect to the remote debugger and attempt to debug a process.
im currently propsoing removing that as part of the removal of eventlet
On Wed, 2024-08-14 at 23:06 +0100, smooney@redhat.com wrote: python this because that they are a distibuted system process is stop and the other contiue partly because it shoudl not be needed
when we dont have enventlet and mainly because it didnt work very well when i last used it.
some ides have supprot fo gevent
gevent is a fork of eventlet which was forked form some of hte libiares in stackless python eventlet converts a single thread python interperater in a cooperative concurent multi userland process.
by the way i forgot to say if you want to debug eventlet mokey patch code adn your using a debugger that supprot gevent it mostly work with eventlet too if you enabel the gevnet supprot in pycham which is disabeld by defaut.
i have had it workk reallly well for months and then not work for years and suddenly start workign agian. so the proablme with usign a debugger with nova is not getting it working initally its keepign it working and setting it up on a new machine if you create a new devstack vm for dev again.
its not hard, its hard to do repatbly, so blogs, docs and even automation tends to bit wroth quickly.
what does that mean , each python function is not operating in a seperate os stackframe, it has heap allcoated stack frames that allow context swtichs between userland thread at any tiem a funnction woudl do blocking io
the reason single step debugging does not work well with gdb is that when you single step the eventlet engine is often going to end up context switching into the midel of another function.
to work around this you just set breakpoint at the line you want to go to next and use continue instead fo single step
one of the rules when using eventlet is that if your going to monkey patch you need to monkey patch everyting early
in addtion too the remote debuging facilites we can disabl mokey patching entirly. you can do that by settign OS_NOVA_DISABLE_EVENTLET_PATCHING=1 https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L87
this will entirly break nova-compute and several other nova process as they have effectivly infinity loops to pool for rpc messags, monitor hyperviors ectra.
if the remote debugger is used it disables patching the thread lib
https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L46
that is prartly why usign the remote debugger is kind fo buggy in
itsself.
the code is not inteded to work that way.
these fucntionlatiy mainly exist to allow debuging tests not the actulaly running code.
if i need to debug test i normaly jsut comment out
https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L83-L89
and run the test in my ides(pycharm,vscodes) test debbuger. i normally code in nano/emacs but is do use ides if i really cant figure out what happening however i genreally start with just adding log lines.
when i first started workign on opentack in 2013 i very much liked using an ide and debugger and i woudl often spend time trying to get nova to work in a debugger.
over tiem i generally found that that was not worth the effort. keepign a devstack deployment working that way was often more work then doing print staement or better writing functional or unit test to repoduce the problem.
you previous mails ask about debuging nova's api.
if you want to do that its one of the easiest thigns to debug. first do _not_ use the wsgi script adding apache of modwsgi is just going to make your life harder.
just disable eventlet monkey patching, by commetiing or usign the env varible and run the nova-api console script directly under pdb or your debugger of choice. that will allow you to basiclly single step debug without any issues as it will not be using eventlet anymore and it has not infinitly loops or perodic tasks so it mostly just works when not using eventlet.
that becase it didnt use eventlet at all in the api until about 2016 and even today it only really uses it in one place in my eventlet removal serise i split nova in to the part that use eventlet directly and the parts that dont in the second path
https://review.opendev.org/c/openstack/nova/+/904424
anything under nova/cmd/eventlet/ needs eventlet today everything under nova/cmd/standalone/ does not.
the approch of doign an api call and following it in a debugger is not a good way in my expericne to learn how somethign like nova really works. it can hellp to a limtied degree but you cant eaislly debug across multipel processes. a minium nova deployment has 4 process typically more.
you can try but each openstack service (nova, neutorn, cinder, ...) is typically compised of several micocservices some like keystone and palcement are just rest apis in front of a db and dont use eventlet at all so are trivial to debug as a result. the rest have non tirival runtime interactions that cant be as eaislly decompsed. becuase of that nova and other project invest in our functional tests to allwo ues to emulate that multi process env and better reason about the code.
the debugging functionality is documetned in the contibutor guide
but we dont really adverstise that because none of the core maintainers use that any more and it has not been maintained for the better part of a decade. since we cant use this type of debugging with our ci system if we cant debug why a failure happens form the ci logs we add more logs because thyat will be useful for future us and operators in production. so our focus is on logging/monitoring for
simply because of the large techdebt that eventlet creates in this aready.
i know nuetorn has had some better look with using debuggers then nova has,
keystone/placmnet are eventlest free adn are just python web apps/rest apis so should "just work" in any python debugger.
nova is prehaps the hardest to debug because of the legacy of the
how this works. too be clear i have ran nova api, nova-conductor, nova scheduler and nova-compute all in 4 ide windows with 4 debuggers at once it was proably 3 year of working on openstack before i understood how to hack that ot work and since i learned how to properly debug without a debuggger, logs, code inspection, unit/fucntioal testing i have nver really need to do that again.
gaining the ablity to run openstack in a debugger simply again, in a
im looking forward to removing eventlet but its still a lot of work. im not sure how helpful this was but this is proably as clsoe to a blog
https://github.com/openstack/nova/blob/master/doc/source/contributor/develop... production rather then for developer eases proejct so its not the best palce to start looking maintainable way is deffienly one of the reason post as you will get.
On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
In almost a decade of reading smooney@redhat.com's Lysdexic posts, this is the longest one I have ever read. No criticism intended at all - none whatsoever. BUT... I am still absolutely Gobsmacked that anyone can possibly write even vaguely working code, if their written English communications are so comprehensively broken. How does anyone get away with ignoring Spelling and Grammar in Code? I love your work Sean, but I do struggle to understand almost everything you type. -- MC On Thu, 15 Aug 2024 at 08:07, <smooney@redhat.com> wrote:
On Thu, 2024-08-15 at 01:49 +0530, engineer2024 wrote:
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc.
Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information.
But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the python way since pdb supports python prompt as well. Of course I do all this on dev systems though...
this is going to be a long responce and its a bit rambely but im goign to provide you with the context and poitn er try and figure out how to do this for your self.
first thing to note is while its possibel it not trivial and the payoff is proably not worth it in the long run. we do not have up to day docs for how to do this because this is not how peole learn, contibute to ro develop nova in genral.
nova does have a remote debug facilitate that is sepreate for the eventlet backdor to enable you to use an ide to debug remotely. that was added around 2012 and has worked on an off to vering degrees since then.
the first thing to understand about nova and several other project is that they are a distibuted system i.e. the applciaiton that compise a given service like nova run in multiple service which are interconnected vai a message bus with in a service and rest apis are exposed for inter service comunication.
when you enable a debugger and stop on a breakpoitn only one of the process is stop and the other contiue to exsculte which means that RPC calls, heathbeats and other asynconos tasks can and will timeout and fail.
you cannot debug a distibuted system in teh same way as you woudl a simple applction where all state is executed in a signle main loop.
the act of observing the internal behavior of the nova api or nova-comptue agent will change its behavior.
nova has suppoort for remote debugging that was added about 12 years ago https://wiki.openstack.org/wiki/Nova/RemoteDebugging https://github.com/openstack/nova/blob/master/nova/debugger.py
with that supprot you can use an idea like pycharm to connect to the remote debugger and attempt to debug a process.
im currently propsoing removing that as part of the removal of eventlet partly because it shoudl not be needed when we dont have enventlet and mainly because it didnt work very well when i last used it.
some ides have supprot fo gevent
gevent is a fork of eventlet which was forked form some of hte libiares in stackless python eventlet converts a single thread python interperater in a cooperative concurent multi userland process.
what does that mean , each python function is not operating in a seperate os stackframe, it has heap allcoated stack frames that allow context swtichs between userland thread at any tiem a funnction woudl do blocking io
the reason single step debugging does not work well with gdb is that when you single step the eventlet engine is often going to end up context switching into the midel of another function.
to work around this you just set breakpoint at the line you want to go to next and use continue instead fo single step
one of the rules when using eventlet is that if your going to monkey patch you need to monkey patch everyting early
in addtion too the remote debuging facilites we can disabl mokey patching entirly. you can do that by settign OS_NOVA_DISABLE_EVENTLET_PATCHING=1 https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L87
this will entirly break nova-compute and several other nova process as they have effectivly infinity loops to pool for rpc messags, monitor hyperviors ectra.
if the remote debugger is used it disables patching the thread lib https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L46
that is prartly why usign the remote debugger is kind fo buggy in itsself. the code is not inteded to work that way.
these fucntionlatiy mainly exist to allow debuging tests not the actulaly running code.
if i need to debug test i normaly jsut comment out https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L83-L89 and run the test in my ides(pycharm,vscodes) test debbuger. i normally code in nano/emacs but is do use ides if i really cant figure out what happening however i genreally start with just adding log lines.
when i first started workign on opentack in 2013 i very much liked using an ide and debugger and i woudl often spend time trying to get nova to work in a debugger.
over tiem i generally found that that was not worth the effort. keepign a devstack deployment working that way was often more work then doing print staement or better writing functional or unit test to repoduce the problem.
you previous mails ask about debuging nova's api.
if you want to do that its one of the easiest thigns to debug. first do _not_ use the wsgi script adding apache of modwsgi is just going to make your life harder.
just disable eventlet monkey patching, by commetiing or usign the env varible and run the nova-api console script directly under pdb or your debugger of choice. that will allow you to basiclly single step debug without any issues as it will not be using eventlet anymore and it has not infinitly loops or perodic tasks so it mostly just works when not using eventlet.
that becase it didnt use eventlet at all in the api until about 2016 and even today it only really uses it in one place in my eventlet removal serise i split nova in to the part that use eventlet directly and the parts that dont in the second path
https://review.opendev.org/c/openstack/nova/+/904424
anything under nova/cmd/eventlet/ needs eventlet today everything under nova/cmd/standalone/ does not.
the approch of doign an api call and following it in a debugger is not a good way in my expericne to learn how somethign like nova really works. it can hellp to a limtied degree but you cant eaislly debug across multipel processes. a minium nova deployment has 4 process typically more.
you can try but each openstack service (nova, neutorn, cinder, ...) is typically compised of several micocservices some like keystone and palcement are just rest apis in front of a db and dont use eventlet at all so are trivial to debug as a result. the rest have non tirival runtime interactions that cant be as eaislly decompsed. becuase of that nova and other project invest in our functional tests to allwo ues to emulate that multi process env and better reason about the code.
the debugging functionality is documetned in the contibutor guide
https://github.com/openstack/nova/blob/master/doc/source/contributor/develop... but we dont really adverstise that because none of the core maintainers use that any more and it has not been maintained for the better part of a decade. since we cant use this type of debugging with our ci system if we cant debug why a failure happens form the ci logs we add more logs because thyat will be useful for future us and operators in production. so our focus is on logging/monitoring for production rather then for developer eases simply because of the large techdebt that eventlet creates in this aready.
i know nuetorn has had some better look with using debuggers then nova has,
keystone/placmnet are eventlest free adn are just python web apps/rest apis so should "just work" in any python debugger.
nova is prehaps the hardest to debug because of the legacy of the proejct so its not the best palce to start looking how this works. too be clear i have ran nova api, nova-conductor, nova scheduler and nova-compute all in 4 ide windows with 4 debuggers at once it was proably 3 year of working on openstack before i understood how to hack that ot work and since i learned how to properly debug without a debuggger, logs, code inspection, unit/fucntioal testing i have nver really need to do that again.
gaining the ablity to run openstack in a debugger simply again, in a maintainable way is deffienly one of the reason im looking forward to removing eventlet but its still a lot of work. im not sure how helpful this was but this is proably as clsoe to a blog post as you will get.
On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
Actually, when I was in my graduate/post graduate studies in computer science, I used to think that whoever is technically smart but unable to communicate in english would find a hard time in the industry. But after starting my career in IT and seeing many brilliant minds, I am convinced that language is not as important as the skill itself. We just need to understand each other in anyway possible, not only in english language. So many of my colleagues have gone on to higher positions in IT now despite their lack of proper english communication. So I think if you are able to express your ideas in any form, thats enough. Yes its important for certain roles like sales , client/customer facing. But not so for engineers. And regarding Sean's typos, I can understand he is quick typing on a mobile device or so, that these catch up. No issues though... On Thu, Aug 15, 2024 at 1:06 PM Mike Carden <mike.carden@gmail.com> wrote:
In almost a decade of reading smooney@redhat.com's Lysdexic posts, this is the longest one I have ever read. No criticism intended at all - none whatsoever.
BUT... I am still absolutely Gobsmacked that anyone can possibly write even vaguely working code, if their written English communications are so comprehensively broken. How does anyone get away with ignoring Spelling and Grammar in Code?
I love your work Sean, but I do struggle to understand almost everything you type.
-- MC
On Thu, 15 Aug 2024 at 08:07, <smooney@redhat.com> wrote:
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc.
Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information.
But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the
On Thu, 2024-08-15 at 01:49 +0530, engineer2024 wrote: python
way since pdb supports python prompt as well. Of course I do all this on dev systems though...
this is going to be a long responce and its a bit rambely but im goign to provide you with the context and poitn er try and figure out how to do this for your self.
first thing to note is while its possibel it not trivial and the payoff is proably not worth it in the long run. we do not have up to day docs for how to do this because this is not how peole learn, contibute to ro develop nova in genral.
nova does have a remote debug facilitate that is sepreate for the eventlet backdor to enable you to use an ide to debug remotely. that was added around 2012 and has worked on an off to vering degrees since then.
the first thing to understand about nova and several other project is that they are a distibuted system i.e. the applciaiton that compise a given service like nova run in multiple service which are interconnected vai a message bus with in a service and rest apis are exposed for inter service comunication.
when you enable a debugger and stop on a breakpoitn only one of the process is stop and the other contiue to exsculte which means that RPC calls, heathbeats and other asynconos tasks can and will timeout and fail.
you cannot debug a distibuted system in teh same way as you woudl a simple applction where all state is executed in a signle main loop.
the act of observing the internal behavior of the nova api or nova-comptue agent will change its behavior.
nova has suppoort for remote debugging that was added about 12 years ago https://wiki.openstack.org/wiki/Nova/RemoteDebugging https://github.com/openstack/nova/blob/master/nova/debugger.py
with that supprot you can use an idea like pycharm to connect to the remote debugger and attempt to debug a process.
im currently propsoing removing that as part of the removal of eventlet partly because it shoudl not be needed when we dont have enventlet and mainly because it didnt work very well when i last used it.
some ides have supprot fo gevent
gevent is a fork of eventlet which was forked form some of hte libiares in stackless python eventlet converts a single thread python interperater in a cooperative concurent multi userland process.
what does that mean , each python function is not operating in a seperate os stackframe, it has heap allcoated stack frames that allow context swtichs between userland thread at any tiem a funnction woudl do blocking io
the reason single step debugging does not work well with gdb is that when you single step the eventlet engine is often going to end up context switching into the midel of another function.
to work around this you just set breakpoint at the line you want to go to next and use continue instead fo single step
one of the rules when using eventlet is that if your going to monkey patch you need to monkey patch everyting early
in addtion too the remote debuging facilites we can disabl mokey patching entirly. you can do that by settign OS_NOVA_DISABLE_EVENTLET_PATCHING=1 https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L87
this will entirly break nova-compute and several other nova process as they have effectivly infinity loops to pool for rpc messags, monitor hyperviors ectra.
if the remote debugger is used it disables patching the thread lib https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L46
that is prartly why usign the remote debugger is kind fo buggy in itsself. the code is not inteded to work that way.
these fucntionlatiy mainly exist to allow debuging tests not the actulaly running code.
if i need to debug test i normaly jsut comment out https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L83-L89 and run the test in my ides(pycharm,vscodes) test debbuger. i normally code in nano/emacs but is do use ides if i really cant figure out what happening however i genreally start with just adding log lines.
when i first started workign on opentack in 2013 i very much liked using an ide and debugger and i woudl often spend time trying to get nova to work in a debugger.
over tiem i generally found that that was not worth the effort. keepign a devstack deployment working that way was often more work then doing print staement or better writing functional or unit test to repoduce the problem.
you previous mails ask about debuging nova's api.
if you want to do that its one of the easiest thigns to debug. first do _not_ use the wsgi script adding apache of modwsgi is just going to make your life harder.
just disable eventlet monkey patching, by commetiing or usign the env varible and run the nova-api console script directly under pdb or your debugger of choice. that will allow you to basiclly single step debug without any issues as it will not be using eventlet anymore and it has not infinitly loops or perodic tasks so it mostly just works when not using eventlet.
that becase it didnt use eventlet at all in the api until about 2016 and even today it only really uses it in one place in my eventlet removal serise i split nova in to the part that use eventlet directly and the parts that dont in the second path
https://review.opendev.org/c/openstack/nova/+/904424
anything under nova/cmd/eventlet/ needs eventlet today everything under nova/cmd/standalone/ does not.
the approch of doign an api call and following it in a debugger is not a good way in my expericne to learn how somethign like nova really works. it can hellp to a limtied degree but you cant eaislly debug across multipel processes. a minium nova deployment has 4 process typically more.
you can try but each openstack service (nova, neutorn, cinder, ...) is typically compised of several micocservices some like keystone and palcement are just rest apis in front of a db and dont use eventlet at all so are trivial to debug as a result. the rest have non tirival runtime interactions that cant be as eaislly decompsed. becuase of that nova and other project invest in our functional tests to allwo ues to emulate that multi process env and better reason about the code.
the debugging functionality is documetned in the contibutor guide
https://github.com/openstack/nova/blob/master/doc/source/contributor/develop... but we dont really adverstise that because none of the core maintainers use that any more and it has not been maintained for the better part of a decade. since we cant use this type of debugging with our ci system if we cant debug why a failure happens form the ci logs we add more logs because thyat will be useful for future us and operators in production. so our focus is on logging/monitoring for production rather then for developer eases simply because of the large techdebt that eventlet creates in this aready.
i know nuetorn has had some better look with using debuggers then nova has,
keystone/placmnet are eventlest free adn are just python web apps/rest apis so should "just work" in any python debugger.
nova is prehaps the hardest to debug because of the legacy of the proejct so its not the best palce to start looking how this works. too be clear i have ran nova api, nova-conductor, nova scheduler and nova-compute all in 4 ide windows with 4 debuggers at once it was proably 3 year of working on openstack before i understood how to hack that ot work and since i learned how to properly debug without a debuggger, logs, code inspection, unit/fucntioal testing i have nver really need to do that again.
gaining the ablity to run openstack in a debugger simply again, in a maintainable way is deffienly one of the reason im looking forward to removing eventlet but its still a lot of work. im not sure how helpful this was but this is proably as clsoe to a blog post as you will get.
On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
On Thu, 2024-08-15 at 17:36 +1000, Mike Carden wrote:
In almost a decade of reading smooney@redhat.com's Lysdexic posts, this is the longest one I have ever read. No criticism intended at all - none whatsoever.
BUT... I am still absolutely Gobsmacked that anyone can possibly write even vaguely working code, if their written English communications are so comprehensively broken. How does anyone get away with ignoring Spelling and Grammar in Code?
I love your work Sean, but I do struggle to understand almost everything you type. no worries short answer is i have more tooling to help when im writing code
if im not in a rush or its a more formal topic i heavily rely on things like grammerly or spellcheckers to fix what i typed. as my typeing speed has increased the things i used to only do when writing i.e. swapping/skipping letters or writing a word other then the one i intended have unfortunately increased as i am now more or less using muscel memory for typing and that allow my dyslxia to propagate to my typing in a way that does not otherwise happen. when im coding i actually type much slower as i am mentally thinking more about the code and my ide will also complain at me. i even added codespell checking to nova recently with pre-commit to prevent future typos form being introduced to the codebase as they do get missed in review. tools like codepoilt can also help. when i use it is more or less like auto correct/spellchecking on steroids then actually for code generation. just be careful if you start a comment with NOTE(sean-k-mooney): i have seen it emulate how i write comment including typos before :) you hopefully wont have that problem but i have started to use it to help me write comments because its mostly less error prone then when is entrily write it by hand. in this case the main reason for all the typos is it was almost midnight and i was tired. the more tiered i am the more this happens, the same if i am distracted. effectively the more i rely on my touch typing ability and dont look a the keyboard the more mistakes i will make as when i read back what i typed I'm not actually reading. i think i am but what I'm actually doing is skiming the text and filling in the blanks form my working memory and not seeing what i typed. when it comes to code that is mitigated because i see the code in my editor, then i see it presented differently wiht git diff and finally after i push it i open it in gerrit and see it presented differently again there. that 3 phase reivew has enough visual different that i will actully read the text instead of rememebering it. the fact that copilot can help mask my dyslxia is actully the main reaon i didnt cancel my yearly description because i does help with that alot. i proably should pay for grammerly premium however. especially since if im not in a rush i used it to write all my specs after the initially draft. when i don't people have issues comprehending them :) and rightly so i sometime wonder how i typed what i typed and didnt notice :) if anyone has recommendation for email clients or plugins that can help let me know. i have been holding of actually using an IDE/Emacs for email but maybe i should. currently i use evolution but i used Thunderbird for years. in case its not obvious i spend about 15 mins actually fixing the typos in this email and im sure i have missed some. i find that incredible tiring but im always looking for ways to make my written communication better, so i welcome any suggestions.
-- MC
On Thu, 15 Aug 2024 at 08:07, <smooney@redhat.com> wrote:
On Thu, 2024-08-15 at 01:49 +0530, engineer2024 wrote:
Thanks to both of you for the response . I try to insert pdb breakpoint at some point in the code files and start the service. Then I issue a nova command to check how the flow works. From then on I use the pdb commands to check the function calls. This to me gives a very convenient way of learning the code connections, since I don't have an idea of how these functions and modules r related. U people as developers have a design to start with and work on from there. So u may know the purpose of each function, class, etc.
Since nova api doesn't support pdb becos of eventlet design, I learned to print those objects and find their information.
But with pdb, it's a smooth flow to inspect the class attributes, their functions, arguments, types etc. I can even test those objects the python way since pdb supports python prompt as well. Of course I do all this on dev systems though...
this is going to be a long responce and its a bit rambely but im goign to provide you with the context and poitn er try and figure out how to do this for your self.
first thing to note is while its possibel it not trivial and the payoff is proably not worth it in the long run. we do not have up to day docs for how to do this because this is not how peole learn, contibute to ro develop nova in genral.
nova does have a remote debug facilitate that is sepreate for the eventlet backdor to enable you to use an ide to debug remotely. that was added around 2012 and has worked on an off to vering degrees since then.
the first thing to understand about nova and several other project is that they are a distibuted system i.e. the applciaiton that compise a given service like nova run in multiple service which are interconnected vai a message bus with in a service and rest apis are exposed for inter service comunication.
when you enable a debugger and stop on a breakpoitn only one of the process is stop and the other contiue to exsculte which means that RPC calls, heathbeats and other asynconos tasks can and will timeout and fail.
you cannot debug a distibuted system in teh same way as you woudl a simple applction where all state is executed in a signle main loop.
the act of observing the internal behavior of the nova api or nova-comptue agent will change its behavior.
nova has suppoort for remote debugging that was added about 12 years ago https://wiki.openstack.org/wiki/Nova/RemoteDebugging https://github.com/openstack/nova/blob/master/nova/debugger.py
with that supprot you can use an idea like pycharm to connect to the remote debugger and attempt to debug a process.
im currently propsoing removing that as part of the removal of eventlet partly because it shoudl not be needed when we dont have enventlet and mainly because it didnt work very well when i last used it.
some ides have supprot fo gevent
gevent is a fork of eventlet which was forked form some of hte libiares in stackless python eventlet converts a single thread python interperater in a cooperative concurent multi userland process.
what does that mean , each python function is not operating in a seperate os stackframe, it has heap allcoated stack frames that allow context swtichs between userland thread at any tiem a funnction woudl do blocking io
the reason single step debugging does not work well with gdb is that when you single step the eventlet engine is often going to end up context switching into the midel of another function.
to work around this you just set breakpoint at the line you want to go to next and use continue instead fo single step
one of the rules when using eventlet is that if your going to monkey patch you need to monkey patch everyting early
in addtion too the remote debuging facilites we can disabl mokey patching entirly. you can do that by settign OS_NOVA_DISABLE_EVENTLET_PATCHING=1 https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L87
this will entirly break nova-compute and several other nova process as they have effectivly infinity loops to pool for rpc messags, monitor hyperviors ectra.
if the remote debugger is used it disables patching the thread lib https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L44-L46
that is prartly why usign the remote debugger is kind fo buggy in itsself. the code is not inteded to work that way.
these fucntionlatiy mainly exist to allow debuging tests not the actulaly running code.
if i need to debug test i normaly jsut comment out https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L83-L89 and run the test in my ides(pycharm,vscodes) test debbuger. i normally code in nano/emacs but is do use ides if i really cant figure out what happening however i genreally start with just adding log lines.
when i first started workign on opentack in 2013 i very much liked using an ide and debugger and i woudl often spend time trying to get nova to work in a debugger.
over tiem i generally found that that was not worth the effort. keepign a devstack deployment working that way was often more work then doing print staement or better writing functional or unit test to repoduce the problem.
you previous mails ask about debuging nova's api.
if you want to do that its one of the easiest thigns to debug. first do _not_ use the wsgi script adding apache of modwsgi is just going to make your life harder.
just disable eventlet monkey patching, by commetiing or usign the env varible and run the nova-api console script directly under pdb or your debugger of choice. that will allow you to basiclly single step debug without any issues as it will not be using eventlet anymore and it has not infinitly loops or perodic tasks so it mostly just works when not using eventlet.
that becase it didnt use eventlet at all in the api until about 2016 and even today it only really uses it in one place in my eventlet removal serise i split nova in to the part that use eventlet directly and the parts that dont in the second path
https://review.opendev.org/c/openstack/nova/+/904424
anything under nova/cmd/eventlet/ needs eventlet today everything under nova/cmd/standalone/ does not.
the approch of doign an api call and following it in a debugger is not a good way in my expericne to learn how somethign like nova really works. it can hellp to a limtied degree but you cant eaislly debug across multipel processes. a minium nova deployment has 4 process typically more.
you can try but each openstack service (nova, neutorn, cinder, ...) is typically compised of several micocservices some like keystone and palcement are just rest apis in front of a db and dont use eventlet at all so are trivial to debug as a result. the rest have non tirival runtime interactions that cant be as eaislly decompsed. becuase of that nova and other project invest in our functional tests to allwo ues to emulate that multi process env and better reason about the code.
the debugging functionality is documetned in the contibutor guide
https://github.com/openstack/nova/blob/master/doc/source/contributor/develop... but we dont really adverstise that because none of the core maintainers use that any more and it has not been maintained for the better part of a decade. since we cant use this type of debugging with our ci system if we cant debug why a failure happens form the ci logs we add more logs because thyat will be useful for future us and operators in production. so our focus is on logging/monitoring for production rather then for developer eases simply because of the large techdebt that eventlet creates in this aready.
i know nuetorn has had some better look with using debuggers then nova has,
keystone/placmnet are eventlest free adn are just python web apps/rest apis so should "just work" in any python debugger.
nova is prehaps the hardest to debug because of the legacy of the proejct so its not the best palce to start looking how this works. too be clear i have ran nova api, nova-conductor, nova scheduler and nova-compute all in 4 ide windows with 4 debuggers at once it was proably 3 year of working on openstack before i understood how to hack that ot work and since i learned how to properly debug without a debuggger, logs, code inspection, unit/fucntioal testing i have nver really need to do that again.
gaining the ablity to run openstack in a debugger simply again, in a maintainable way is deffienly one of the reason im looking forward to removing eventlet but its still a lot of work. im not sure how helpful this was but this is proably as clsoe to a blog post as you will get.
On Thu, 15 Aug 2024, 01:04 Jeremy Stanley, <fungi@yuggoth.org> wrote:
On 2024-08-14 22:38:43 +0530 (+0530), engineer2024 wrote:
How many more releases are you planning to include eventlet in them? Seems they are removing any support for further development according to their official website. So, is the community thinking of any alternatives? [...]
Just to expand on the first part of Sean's reply, most of what you want to know is probably answered here:
https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html
-- Jeremy Stanley
On 2024-08-15 11:57:18 +0100 (+0100), smooney@redhat.com wrote: [...]
tools like codepoilt can also help. when i use it is more or less like auto correct/spellchecking on steroids then actually for code generation. just be careful if you start a comment with NOTE(sean-k-mooney): i have seen it emulate how i write comment including typos before :) [...]
This shouldn't be funny, but it really, really is. Sounds like Copilot is becoming a fan of your personal writing style!
in case its not obvious i spend about 15 mins actually fixing the typos in this email and im sure i have missed some. i find that incredible tiring but im always looking for ways to make my written communication better, so i welcome any suggestions. [...]
I mostly just want to say that I find you inspiring. I've struggled with dyslexia all my life, and am sometimes so self-conscious of it and embarrassed by my errors that it's debilitating. My main coping mechanism is to read, re-read, and re-re-read what I write, put it aside, come back and read it again... even for simple commit messages or mailing list replies. The impact of this on my efficiency is, unfortunately, substantial. That you've found a balance to maintain your throughput while still being understood makes you a hero in my eyes. I'm similarly awed by all of you in our community who are communicating in English even though it's not your first language. English is my only language, one I've spoken and written in almost as long as I've been alive, and even I find it a struggle. We each struggle with something that makes communication hard, and what I really love is that, with some effort and accommodation, we do still manage to get our points across in order to collaborate on one of the most influential software projects of all time. That's truly amazing. -- Jeremy Stanley
On 8/15/24 03:57, smooney@redhat.com wrote: [...]
if anyone has recommendation for email clients or plugins that can help let me know. i have been holding of actually using an IDE/Emacs for email but maybe i should. currently i use evolution but i used Thunderbird for years. in case its not obvious i spend about 15 mins actually fixing the typos in this email and im sure i have missed some. i find that incredible tiring but im always looking for ways to make my written communication better, so i welcome any suggestions.
I have been using Thunderbird (I use the Beta version) for some years and I like it. I don't use any add-ons for spelling or grammar but there are several available: https://addons.thunderbird.net/en-US/thunderbird/search/?q=grammar The only gripe I have about Thunderbird is the lack of dead simple automatic dark theme for the message reading window (the rest of the UI does dark mode fine). I like to use light mode during the day and dark mode at night. I have tried DarkReader and it was OK but too clunky for me. For reading plain text I just installed Dark Plain Text which handles automatic dark mode for reading plain text and it looks good but obviously doesn't cover HTML. -melwitt
Since this thread got off topic, I feel less guilty about straying with it... On 16/8/24 04:15, melanie witt wrote:
On 8/15/24 03:57, smooney@redhat.com wrote: [...]
Firstly, gratitude for snipping. Since we're talking about things that happen on mailing lists, why is it that many here have held firmly onto the good old standard for responding in threaded context instead of top-posting, but at the same time, quote the entire message? When I have many emails to go through, if I have to scroll past a heap of quoted text I am far less likely to spend the time to bother with it, unless it's a topic I really want to read - not great for general information gathering. That's key, because there are heaps of useful nuggets on a list like this one. I actually prefer top-posting to this, because at least it is only the 'second' standard, rather than this third one.
The only gripe I have about Thunderbird is the lack of dead simple automatic dark theme for the message reading window (the rest of the UI
True for me too - I use dark mode everywhere I can, all the time.
does dark mode fine). I like to use light mode during the day and dark mode at night. I have tried DarkReader and it was OK but too clunky for
Same about DarkReader. I dropped the equivalent in Firefox too after it got clunky for me.
me. For reading plain text I just installed Dark Plain Text which handles automatic dark mode for reading plain text and it looks good but obviously doesn't cover HTML.
Oh wow, you've got me another step closer to the solution. Thanks! Greg.
On Fri, 2024-08-16 at 13:53 +0800, Gregory Orange wrote:
Since this thread got off topic, I feel less guilty about straying with it...
On 16/8/24 04:15, melanie witt wrote:
On 8/15/24 03:57, smooney@redhat.com wrote: [...]
Firstly, gratitude for snipping.
Since we're talking about things that happen on mailing lists, why is it that many here have held firmly onto the good old standard for responding in threaded context instead of top-posting, but at the same time, quote the entire message?
for me this is my prefernce to keep responses in context. if you top post it make it much harder to have several peopel interact as there may be several sub disucssion happening in the same email at once. we have a https://wiki.openstack.org/wiki/MailingListEtiquette document in that we document this in the replies sub section https://wiki.openstack.org/wiki/MailingListEtiquette#Replies and use the interleave style https://en.wikipedia.org/wiki/Posting_style#Interleaved_style by default we do sometimes drop parts of the original mail if not relevent but not always
When I have many emails to go through, if I have to scroll past a heap of quoted text I am far less likely to spend the time to bother with it, unless it's a topic I really want to read - not great for general information gathering. That's key, because there are heaps of useful nuggets on a list like this one.
I actually prefer top-posting to this, because at least it is only the 'second' standard, rather than this third one.
well the documented standard that we expect peoole to use on the list is the interleave style in plain text. but ill admit i also dont follow all of the rules. i really hate the 72 character line wrapping so i dont follow that. i configure wrapping at 120 instead.
The only gripe I have about Thunderbird is the lack of dead simple automatic dark theme for the message reading window (the rest of the UI
True for me too - I use dark mode everywhere I can, all the time.
same, it really does help reduce eyestrain epically as monitors have gotten brighter.
does dark mode fine). I like to use light mode during the day and dark mode at night. I have tried DarkReader and it was OK but too clunky for
Same about DarkReader. I dropped the equivalent in Firefox too after it got clunky for me.
im still using it but have noticed the slow down. for me its still worth not having as much eye strain issues but it can take a while for some things to render.
me. For reading plain text I just installed Dark Plain Text which handles automatic dark mode for reading plain text and it looks good but obviously doesn't cover HTML.
Oh wow, you've got me another step closer to the solution. Thanks!
ya that i did not know about either. i have been putting of moving back to Thunderbird for quite a while but it might be worth considering. evolution has this habit of deferring work and trying ot update folders only when you click on them that mainly problematic for calendar invites as they dont actully show up in its calandar until you click on the invite.
Greg.
On 2024-08-16 12:31:56 +0100 (+0100), smooney@redhat.com wrote: [...]
i have been putting of moving back to Thunderbird for quite a while but it might be worth considering.
evolution has this habit of deferring work and trying ot update folders only when you click on them that mainly problematic for calendar invites as they dont actully show up in its calandar until you click on the invite.
I suppose we all find different ways to keep things as readable as possible. I do basically everything in 80x25 (really 80x24 with a status line) text terminals, light text on a black background. I spent so many years on old "green screen" serial terminals that it's far faster for me to read. I've never fully understood why "dark mode" support is so hard to come by, my applications have all been that way for more than 4 decades already. For E-mail I still use good ol' mutt (well, mutt-ng) set to raise a silent bell in the tmux window where it runs when messages arrive. For calendaring, I use remind in combination with wyrd configured to ping the silent bell in the tmux window where it's running so that I also get a tab highlight in my status line for that whenever I have meetings starting and such. These share the same tmux session with my IRC client (weechat), and I multi-attach to all of it over mosh from computers in different rooms of the house, netbooks, tablets, my Debian-based smart phone, et cetera. This allows me to roam from room to room, one device to another, from a parking lot to a barstool to an airplane seat, all without losing context. This is important to me because resetting or switching context is one of my biggest cognitive challenges. -- Jeremy Stanley
80x25 with tmux... thought I'm the only dinosaur survived 😋 ________________________________ From: Jeremy Stanley Sent: Friday, August 16, 2024 06:30 AM To: openstack-discuss@lists.openstack.org Subject: Re: sean speak and you! was Re: Eventlet and debugging On 2024-08-16 12:31:56 +0100 (+0100), smooney@redhat.com wrote: [...]
i have been putting of moving back to Thunderbird for quite a while but it might be worth considering.
evolution has this habit of deferring work and trying ot update folders only when you click on them that mainly problematic for calendar invites as they dont actully show up in its calandar until you click on the invite.
I suppose we all find different ways to keep things as readable as possible. I do basically everything in 80x25 (really 80x24 with a status line) text terminals, light text on a black background. I spent so many years on old "green screen" serial terminals that it's far faster for me to read. I've never fully understood why "dark mode" support is so hard to come by, my applications have all been that way for more than 4 decades already. For E-mail I still use good ol' mutt (well, mutt-ng) set to raise a silent bell in the tmux window where it runs when messages arrive. For calendaring, I use remind in combination with wyrd configured to ping the silent bell in the tmux window where it's running so that I also get a tab highlight in my status line for that whenever I have meetings starting and such. These share the same tmux session with my IRC client (weechat), and I multi-attach to all of it over mosh from computers in different rooms of the house, netbooks, tablets, my Debian-based smart phone, et cetera. This allows me to roam from room to room, one device to another, from a parking lot to a barstool to an airplane seat, all without losing context. This is important to me because resetting or switching context is one of my biggest cognitive challenges. -- Jeremy Stanley
On 16/8/24 19:31, smooney@redhat.com wrote:
if you top post it make it much harder to have several peopel interact as there may be several sub disucssion happening in the same email at once.
Agreed.
in that we document this in the replies sub section
https://wiki.openstack.org/wiki/MailingListEtiquette#Replies
Yes, trimming! https://wiki.openstack.org/wiki/MailingListEtiquette#Trimming
participants (7)
-
engineer2024
-
Gregory Orange
-
Jeremy Stanley
-
melanie witt
-
Mike Carden
-
smooney@redhat.com
-
Tony Liu