<div dir="ltr">
<div style="text-size-adjust: auto;">Hi all,</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">please let me re-surface the discussion, </div><div style="text-size-adjust: auto;">and propose StackStorm as [a starting point for] a Serverless framework for OpenStack. </div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">StackStorm [0] is an open source event-driven automation platform, built around Mistral and adds “before” and “after” - (triggers on events, and rules that maps triggers to actions) and after (easy to add actions with auto-generated API) [1].</div><div style="text-size-adjust: auto;">It is built for devops automation and often used to automate OpenStack [2]. Recently, we saw it used as a DIY serverless framework, as it has the parts of Serverless stack discussed here: “Lambda”, API Gateway, event sources (not just timers and web hooks but any other events as triggers), and workflows/Step Functions with Mistral - as workflows become recognizable part of serverless.</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">Comparing with OpenWhisk, StackStorm brings the same concepts and functionality, but does it on OpenStack technology stack: Python, RabbitMQ, Pecan, eventlets, Oslo config & utils, and a fair amount of same 3rd party dependencies.</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">StackStorm is mature (3 years old), heavily used (~3,000 installations / month), have rich set of existing integrations [3]. StackStorm team has been a part of OpenStack community and Mistral contributors from the outset.</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">I hate that this is coming out as a plug, but I’d hate it even more if there is a match, and we miss it.</div><div style="text-size-adjust: auto;">Let’s decide on merits. The detailed discussion takes more writing…</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">How about we use the Boston summit to open the conversation? Here is what we can do:</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">1) My talk “Serverless on OpenStack with StackStorm and Mistral” Wed May10 9am can serve as an introduction: <a href="https://goo.gl/2ZUJFK">https://goo.gl/2ZUJFK</a></div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">2) Let’s make time at the summit with those who’re interested AND in Boston, do in-depth technical, architecture, and merits discussion, and share the findings here. </div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">3) If the community believes the idea worth further consideration, we take it from there.</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">If you’re interested to participate, reply here and/or contact me directly.</div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">[0] StackStorm: <a href="https://github.com/StackStorm/st2">https://github.com/StackStorm/st2</a></div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">[1] Slides from OpenStack Barcelona where I explained the relations between Mistral and StackStorm to Mistral community. <a href="https://www.slideshare.net/DmitriZimine/mistral-and-stackstorm">https://www.slideshare.net/DmitriZimine/mistral-and-stackstorm</a></div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">[2] E.g. Cybera, or Symantec: </div><div style="text-size-adjust: auto;"> - <a href="https://www.openstack.org/videos/video/sleep-better-at-night-openstack-cloud-auto-healing">https://www.openstack.org/videos/video/sleep-better-at-night-openstack-cloud-auto-healing</a></div><div style="text-size-adjust: auto;"> - <a href="https://www.mirantis.com/blog/auto-remediation-making-an-openstack-cloud-self-healing/">https://www.mirantis.com/blog/auto-remediation-making-an-openstack-cloud-self-healing/</a> </div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;">[3] Integration points (aka packs) <a href="https://exchange.stackstorm.org/">https://exchange.stackstorm.org/</a></div><div style="text-size-adjust: auto;"><br></div><div style="text-size-adjust: auto;"><br></div><div class="gmail_extra" style="text-size-adjust: auto;"><br><div class="gmail_quote">On Sat, May 6, 2017 at 12:25 PM, Dmitri Zimine <span dir="ltr"><<a href="mailto:dzimine@brocade.com" target="_blank">dzimine@brocade.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:calibri,sans-serif"><div><div><br></div><div><div id="gmail-m_-1347978350746018285MAC_OUTLOOK_SIGNATURE"></div></div></div><div><br></div><span id="gmail-m_-1347978350746018285OLK_SRC_BODY_SECTION"><div style="font-family:calibri;font-size:12pt;border-width:1pt medium medium;border-style:solid none none;padding:3pt 0in 0in;border-top-color:rgb(181,196,223)"><span style="font-weight:bold">From: </span>Lingxian Kong <<a href="mailto:anlin.kong@gmail.com" target="_blank">anlin.kong@gmail.com</a>><br><span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><br><span style="font-weight:bold">Date: </span>Wednesday, November 2, 2016 at 6:20 PM<br><span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><br><span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [FaaS] Function as a service in OpenStack<br></div><div><br></div><div><div dir="ltr"><div style="font-family:"trebuchet ms",sans-serif"><span style="font-family:arial,sans-serif">On Thu, Nov 3, 2016 at 10:44 AM, Zane Bitter </span><span dir="ltr" style="font-family:arial,sans-serif"><<a href="mailto:zbitter@redhat.com" target="_blank">zbitter@redhat.com</a>></span><span style="font-family:arial,sans-serif"> wrote:</span></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">This is a really interesting space. There seems to be two main use cases for Lambda that are probably worth talking about separately:<br><br>The first is for just Lambda alone. You can use it to provide some glue logic between the other AWS services, so you can trigger off various events (e.g. S3 notifications) and write a little bit of conditioning logic that transforms the data and dispatches it to other services (e.g. DynamoDB). This one is particularly interesting to me, and in fact we can support parts of this in OpenStack already[1] because Mistral's functionality is equivalent to something like SWS + parts of Lambda. (Specifically, Mistral can do the data dispatch easily enough, but any data transformation has to be done in YAQL, which is a pretty high bar compared to just writing some code in a language of your choosing.)<br></blockquote><div><br></div><div><div style="font-family:"trebuchet ms",sans-serif">There is still one thing missing in Mistral (maybe it should not be). After receieving events from Aodh or Zaqar, what if user just wants to trigger some scripts under his/her management, rather than just invoking openstack services api? Although actions are pluggable in Mistral, but in this case it's definitely not an easy thing as just writing an customized action, unless Mistral could include such capatility in its scope which I think it too heavy for Mistral to mange such things by itself. So, FaaS will be the right answer in this case, and it will also be add-on part to empower Mistral to do more things.</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>The second one is Lambda + the API Gateway, which allows you to have web requests act as triggers, so that you can effectively treat it as a PaaS and build an entire web app by stringing together Lambda functions and the various other services (S3, DynamoDB, &c.). On the face of it this sounds to me like a gimmicky way of deploying an unmaintainable mess. Naturally this is the one receiving all of the attention, which shows how much I know :D</blockquote><div><br></div><div style="font-family:"trebuchet ms",sans-serif">I also don't think this one is attractive to me, Lambda is especially powerful when it's used together with other AWS services(S3, DynamoDB, Kinesis Streams, etc).</div><div style="font-family:"trebuchet ms",sans-serif"></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_-1347978350746018285gmail-"><br></span>I definitely don't think we should try to reimplement this from scratch in OpenStack. IMHO if we're going to add FaaS capabilities we should re-use some existing project (like OpenWhisk), even if we have to write our own native API over the top of it.<br><br>The things we'd really want it to do would be:<br><br>* Authenticate against Keystone,<br>* Provide Keystone credentials for the user-supplied functions it runs to access (probably using Keystone trusts), and<br>* Connect to existing OpenStack sources of events, which hopefully means Zaqar queues<br><br>Which sounds challenging to integrate with an existing standalone project, though still not as bad as writing an equivalent from scratch.<br><br>TBH I think the appeal, at least for the FaaS-as-a-PaaS (aka #serverless) crowd, is going to be pretty limited until such time as we have an equivalent of DynamoDB in OpenStack. (i.e. no time soon, since the MagnetoDB project is goneburger.) The idea of FaaS is to make the unit of compute power that you're paying for (a) as fine-grained as possible, and (b) scalable to infinity. Swift provides the same thing for storage (Nova:FaaS::Cinder:Swift). What we don't have is the equivalent for a database, there's only Trove where you're paying for a VM-sized chunk at a minimum and scaling up in units of VM-sized chunks until you reach the limit of how many VMs can communicate with each other and still get any work done. Not many web apps can get by without a database, so that largely negates the purpose to my mind, since the database will likely both dominate costs at the low end and put the upper limit on scale at the high end.<br></blockquote><div><br></div><div><div style="font-family:"trebuchet ms",sans-serif">Yeah, I agree with you that more things are needed so that FaaS-like stuff could be used appropriately and ideally, we can't get everything ready on day 1, that's how we do things, from simple to complex, isn't it? </div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>cheers,<br>Zane.<br><br>[1] <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openstack.org_videos_video_building-2Dself-2Dhealing-2Dapplications-2Dwith-2Daodh-2Dzaqar-2Dand-2Dmistral&d=DQMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=ryJXO1JFGuchdwZth4D1J3BpyUYZKSQbrClOec90DjQ&m=sybVYkjPj_Gx0CGJ3jnrlrXHV39hTVkuiC6azF9cfdo&s=g4NR-dY4uSTnfegqb7Mw0w82_X__ijhWnAW4xaZSWNE&e=" rel="noreferrer" target="_blank">https://www.openstack.org/vide<wbr>os/video/building-self-healing<wbr>-applications-with-aodh-zaqar-<wbr>and-mistral</a><div class="gmail-m_-1347978350746018285gmail-HOEnZb"><div class="gmail-m_-1347978350746018285gmail-h5"><br><br><br>______________________________<wbr>______________________________<wbr>______________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__OpenStack-2Ddev-2Drequest-40lists.openstack.org-3Fsubject-3Aunsubscribe&d=DQMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=ryJXO1JFGuchdwZth4D1J3BpyUYZKSQbrClOec90DjQ&m=sybVYkjPj_Gx0CGJ3jnrlrXHV39hTVkuiC6azF9cfdo&s=3_Ocn6IDyq92hVuhr2zMy46n0iRsJexSm-G0OMgzjCI&e=" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DQMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=ryJXO1JFGuchdwZth4D1J3BpyUYZKSQbrClOec90DjQ&m=sybVYkjPj_Gx0CGJ3jnrlrXHV39hTVkuiC6azF9cfdo&s=OZ-TQQcMlp4BKauIyKjeAia29YNDs-WATzDyxaGgBbY&e=" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br></div></div></blockquote></div><br></div></div></div></span></div></blockquote></div></div><p class="gmail-p1"><span class="gmail-s1"></span></p><p class="gmail-p1"><span class="gmail-s1"></span></p><p class="gmail-p1"><span class="gmail-s1"></span></p><p class="gmail-p1"><span class="gmail-s1"></span></p><p class="gmail-p1"><span class="gmail-s1"></span>
</p></div>