<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><title>Re: [Openstack] eventlet weirdness</title><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-GB link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>That sounds a bit over complicated to me – Having a string of tasks sounds like you still have to think about what the concurrency is within each step.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>There is already a good abstraction around the context of each operation – they just (I know – big just) need to be running in something that maps to kernel threads rather than user space ones.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>All I really want to is to allow more than one action to run at the same time. So if I have two requests to create a snapshot, why can’t they both run at the same time and still allow other things to happen ? I have all these cores sitting in my compute node that there that could be used, but I’m still having to think like a punch-card programmer submitting batch jobs to the mainframe ;-)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Right now creating snapshots is pretty close to a DoS attack on a compute node.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Joshua Harlow [mailto:harlowja@yahoo-inc.com] <br><b>Sent:</b> 02 March 2012 19:23<br><b>To:</b> Day, Phil; Chris Behrens<br><b>Cc:</b> openstack<br><b>Subject:</b> Re: [Openstack] eventlet weirdness<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>So a thought I had was that say if the <i>design </i>of a component forces as part of its design the ability to be ran with threads or with eventlet or with processes. <br><br>Say if u break everything up into tasks (where a task would produce some output/result/side-effect).<br>A set of tasks could complete some action (ie, create a vm).<br>Subtasks could be the following: <br> 0. Validate credentials<br> 1. Get the image<br> 2. Call into libvirt<br> 3. ...<br><br>These “tasks”, if constructed in a way that makes them stateless, and then could be chained together to form an action, then that action could be given say to a threaded “engine” that would know how to execute those tasks with threads, or it could be given to an eventlet “engine” that would do the same with evenlet pool/greenthreads/coroutings, or with processes (and so on). This could be one way the design of your code abstracts that kind of execution (where eventlet is abstracted away from the actual work being done, instead of popping up in calls to sleep(0), ie the leaky abstraction). <br><br>On 3/2/12 11:08 AM, "Day, Phil" <<a href="philip.day@hp.com">philip.day@hp.com</a>> wrote:</span><o:p></o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>I didn't say it was pretty - Given the choice I'd much rather have a threading model that really did concurrency and pre-emption all the right places, and it would be really cool if something managed the threads that were started so that is a second conflicting request was received it did some proper tidy up or blocking rather than just leaving the race condition to work itself out (then we wouldn't have to try and control it by checking vm_state).<br><br>However ... In the current code base where we only have user space based eventlets, with no pre-emption, and some activities that need to be prioritised then forcing pre-emption with a sleep(0) seems a pretty small bit of untidy. And it works now without a major code refactor.<br><br>Always open to other approaches ...<br><br>Phil<br><br><br>-----Original Message-----<br>From: <a href="openstack-bounces+philip.day=hp.com@lists.launchpad.net">openstack-bounces+philip.day=hp.com@lists.launchpad.net</a> [<a href="mailto:openstack-bounces+philip.day=hp.com@lists.launchpad.net">mailto:openstack-bounces+philip.day=hp.com@lists.launchpad.net</a>] On Behalf Of Chris Behrens<br>Sent: 02 March 2012 19:00<br>To: Joshua Harlow<br>Cc: openstack; Chris Behrens<br>Subject: Re: [Openstack] eventlet weirdness<br><br>It's not just you<br><br><br>On Mar 2, 2012, at 10:35 AM, Joshua Harlow wrote:<br><br>> Does anyone else feel that the following seems really "dirty", or is it just me.<br>><br>> "adding a few sleep(0) calls in various places in the Nova codebase<br>> (as was recently added in the _sync_power_states() periodic task) is<br>> an easy and simple win with pretty much no ill side-effects. :)"<br>><br>> Dirty in that it feels like there is something wrong from a design point of view.<br>> Sprinkling "sleep(0)" seems like its a band-aid on a larger problem imho.<br>> But that's just my gut feeling.<br>><br>> :-(<br>><br>> On 3/2/12 8:26 AM, "Armando Migliaccio" <<a href="Armando.Migliaccio@eu.citrix.com">Armando.Migliaccio@eu.citrix.com</a>> wrote:<br>><br>> I knew you'd say that :P<br>><br>> There you go: <a href="https://bugs.launchpad.net/nova/+bug/944145">https://bugs.launchpad.net/nova/+bug/944145</a><br>><br>> Cheers,<br>> Armando<br>><br>> > -----Original Message-----<br>> > From: Jay Pipes [<a href="mailto:jaypipes@gmail.com">mailto:jaypipes@gmail.com</a>]<br>> > Sent: 02 March 2012 16:22<br>> > To: Armando Migliaccio<br>> > Cc: <a href="openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>> > Subject: Re: [Openstack] eventlet weirdness<br>> ><br>> > On 03/02/2012 10:52 AM, Armando Migliaccio wrote:<br>> > > I'd be cautious to say that no ill side-effects were introduced. I<br>> > > found a<br>> > race condition right in the middle of sync_power_states, which I<br>> > assume was exposed by "breaking" the task deliberately.<br>> ><br>> > Such a party-pooper! ;)<br>> ><br>> > Got a link to the bug report for me?<br>> ><br>> > Thanks!<br>> > -jay<br>><br>> _______________________________________________<br>> Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>> Post to : <a href="openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>> Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>> More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br>><br>> _______________________________________________<br>> Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>> Post to : <a href="openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>> Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>> More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br><br><br>_______________________________________________<br>Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>Post to : <a href="openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a></span><o:p></o:p></p></div></body></html>