Open Stack

 Hi all,

please let me re-surface the discussion,
and propose StackStorm as [a starting point for] a Serverless framework for
OpenStack.

StackStorm [0] is an open source event-driven automation platform, built
around Mistral and adds “before” and “after” - (triggers on events, and
rules that maps triggers to actions) and after (easy to add actions with
auto-generated API) [1].
It is built for devops automation and often used to automate OpenStack [2].
Recently, we saw it used as a DIY serverless framework, as it has the parts
of Serverless stack discussed here: “Lambda”, API Gateway, event sources
(not just timers and web hooks but any other events as triggers), and
workflows/Step Functions with Mistral - as workflows become recognizable
part of serverless.

Comparing with OpenWhisk, StackStorm brings the same concepts and
functionality, but does it on OpenStack technology stack: Python, RabbitMQ,
Pecan, eventlets, Oslo config & utils, and a fair amount of same 3rd party
dependencies.

StackStorm is mature (3 years old), heavily used (~3,000 installations /
month), have rich set of existing integrations [3]. StackStorm team has
been a part of OpenStack community and Mistral contributors from the outset.

I hate that this is coming out as a plug, but I’d hate it even more if
there is a match, and we miss it.
Let’s decide on merits. The detailed discussion takes more writing…

How about we use the Boston summit to open the conversation? Here is what
we can do:

1) My talk “Serverless on OpenStack with StackStorm and Mistral” Wed May10
9am can serve as an introduction: https://goo.gl/2ZUJFK

2) Let’s make time at the summit with those who’re interested AND in
Boston, do in-depth technical, architecture, and merits discussion, and
share the findings here.

3) If the community believes the idea worth further consideration, we take
it from there.

If you’re interested to participate, reply here and/or contact me directly.

[0] StackStorm: https://github.com/StackStorm/st2

[1] Slides from OpenStack Barcelona where I explained the relations between
Mistral and StackStorm to Mistral community.
https://www.slideshare.net/DmitriZimine/mistral-and-stackstorm

[2] E.g. Cybera, or Symantec:
     -
https://www.openstack.org/videos/video/sleep-better-at-night-openstack-cloud-auto-healing
     -
https://www.mirantis.com/blog/auto-remediation-making-an-openstack-cloud-self-healing/


[3] Integration points (aka packs) https://exchange.stackstorm.org/



On Sat, May 6, 2017 at 12:25 PM, Dmitri Zimine <dzimine at brocade.com> wrote:

>
>
> From: Lingxian Kong <anlin.kong at gmail.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Date: Wednesday, November 2, 2016 at 6:20 PM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [FaaS] Function as a service in OpenStack
>
> On Thu, Nov 3, 2016 at 10:44 AM, Zane Bitter <zbitter at redhat.com> wrote:
>
>> This is a really interesting space. There seems to be two main use cases
>> for Lambda that are probably worth talking about separately:
>>
>> The first is for just Lambda alone. You can use it to provide some glue
>> logic between the other AWS services, so you can trigger off various events
>> (e.g. S3 notifications) and write a little bit of conditioning logic that
>> transforms the data and dispatches it to other services (e.g. DynamoDB).
>> This one is particularly interesting to me, and in fact we can support
>> parts of this in OpenStack already[1] because Mistral's functionality is
>> equivalent to something like SWS + parts of Lambda. (Specifically, Mistral
>> can do the data dispatch easily enough, but any data transformation has to
>> be done in YAQL, which is a pretty high bar compared to just writing some
>> code in a language of your choosing.)
>>
>
> ​There is still one thing missing in Mistral​ (maybe it should not be).
> After receieving events from Aodh or Zaqar, what if user just wants to
> trigger some scripts under his/her management, rather than just invoking
> openstack services api? Although actions are pluggable in Mistral, but in
> this case it's definitely not an easy thing as just writing an customized
> action, unless Mistral could include such capatility in its scope which I
> think it too heavy for Mistral to mange such things by itself. So, FaaS
> will be the right answer in this case, and it will also be add-on part to
> empower Mistral to do more things.
>
>
>>
>> The second one is Lambda + the API Gateway, which allows you to have web
>> requests act as triggers, so that you can effectively treat it as a PaaS
>> and build an entire web app by stringing together Lambda functions and the
>> various other services (S3, DynamoDB, &c.). On the face of it this sounds
>> to me like a gimmicky way of deploying an unmaintainable mess. Naturally
>> this is the one receiving all of the attention, which shows how much I know
>> :D
>
>
> ​I also don't think this one is attractive to me, Lambda is especially
> powerful when it's used together with other AWS services(S3,
> DynamoDB, Kinesis Streams, etc).
> ​​
>
>>
>> I definitely don't think we should try to reimplement this from scratch
>> in OpenStack. IMHO if we're going to add FaaS capabilities we should re-use
>> some existing project (like OpenWhisk), even if we have to write our own
>> native API over the top of it.
>>
>> The things we'd really want it to do would be:
>>
>> * Authenticate against Keystone,
>> * Provide Keystone credentials for the user-supplied functions it runs to
>> access (probably using Keystone trusts), and
>> * Connect to existing OpenStack sources of events, which hopefully means
>> Zaqar queues
>>
>> Which sounds challenging to integrate with an existing standalone
>> project, though still not as bad as writing an equivalent from scratch.
>>
>> TBH I think the appeal, at least for the FaaS-as-a-PaaS (aka #serverless)
>> crowd, is going to be pretty limited until such time as we have an
>> equivalent of DynamoDB in OpenStack. (i.e. no time soon, since the
>> MagnetoDB project is goneburger.) The idea of FaaS is to make the unit of
>> compute power that you're paying for (a) as fine-grained as possible, and
>> (b) scalable to infinity. Swift provides the same thing for storage
>> (Nova:FaaS::Cinder:Swift). What we don't have is the equivalent for a
>> database, there's only Trove where you're paying for a VM-sized chunk at a
>> minimum and scaling up in units of VM-sized chunks until you reach the
>> limit of how many VMs can communicate with each other and still get any
>> work done. Not many web apps can get by without a database, so that largely
>> negates the purpose to my mind, since the database will likely both
>> dominate costs at the low end and put the upper limit on scale at the high
>> end.
>>
>
> ​Yeah, I agree with you that more things are needed so that FaaS-like
> stuff could be used appropriately and ideally, we can't get everything
> ready on day 1, that's how we do things,  from simple to complex, isn't
> it?
>
>
>
>>
>> cheers,
>> Zane.
>>
>> [1] https://www.openstack.org/videos/video/building-self-healing
>> -applications-with-aodh-zaqar-and-mistral
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openstack.org_videos_video_building-2Dself-2Dhealing-2Dapplications-2Dwith-2Daodh-2Dzaqar-2Dand-2Dmistral&d=DQMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=ryJXO1JFGuchdwZth4D1J3BpyUYZKSQbrClOec90DjQ&m=sybVYkjPj_Gx0CGJ3jnrlrXHV39hTVkuiC6azF9cfdo&s=g4NR-dY4uSTnfegqb7Mw0w82_X__ijhWnAW4xaZSWNE&e=>
>>
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__OpenStack-2Ddev-2Drequest-40lists.openstack.org-3Fsubject-3Aunsubscribe&d=DQMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=ryJXO1JFGuchdwZth4D1J3BpyUYZKSQbrClOec90DjQ&m=sybVYkjPj_Gx0CGJ3jnrlrXHV39hTVkuiC6azF9cfdo&s=3_Ocn6IDyq92hVuhr2zMy46n0iRsJexSm-G0OMgzjCI&e=>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DQMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=ryJXO1JFGuchdwZth4D1J3BpyUYZKSQbrClOec90DjQ&m=sybVYkjPj_Gx0CGJ3jnrlrXHV39hTVkuiC6azF9cfdo&s=OZ-TQQcMlp4BKauIyKjeAia29YNDs-WATzDyxaGgBbY&e=>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170506/d35296ac/attachment.html>