Open Stack

Would Kubernetes be a good fit? It might be possible to hook up a Zaqar queue to submit k8s Jobs?

Thanks,
Kevin
________________________________
From: Lingxian Kong [anlin.kong at gmail.com]
Sent: Wednesday, November 02, 2016 6:20 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [FaaS] Function as a service in OpenStack

On Thu, Nov 3, 2016 at 10:44 AM, Zane Bitter <zbitter at redhat.com<mailto:zbitter at redhat.com>> wrote:
This is a really interesting space. There seems to be two main use cases for Lambda that are probably worth talking about separately:

The first is for just Lambda alone. You can use it to provide some glue logic between the other AWS services, so you can trigger off various events (e.g. S3 notifications) and write a little bit of conditioning logic that transforms the data and dispatches it to other services (e.g. DynamoDB). This one is particularly interesting to me, and in fact we can support parts of this in OpenStack already[1] because Mistral's functionality is equivalent to something like SWS + parts of Lambda. (Specifically, Mistral can do the data dispatch easily enough, but any data transformation has to be done in YAQL, which is a pretty high bar compared to just writing some code in a language of your choosing.)

​There is still one thing missing in Mistral​ (maybe it should not be). After receieving events from Aodh or Zaqar, what if user just wants to trigger some scripts under his/her management, rather than just invoking openstack services api? Although actions are pluggable in Mistral, but in this case it's definitely not an easy thing as just writing an customized action, unless Mistral could include such capatility in its scope which I think it too heavy for Mistral to mange such things by itself. So, FaaS will be the right answer in this case, and it will also be add-on part to empower Mistral to do more things.


The second one is Lambda + the API Gateway, which allows you to have web requests act as triggers, so that you can effectively treat it as a PaaS and build an entire web app by stringing together Lambda functions and the various other services (S3, DynamoDB, &c.). On the face of it this sounds to me like a gimmicky way of deploying an unmaintainable mess. Naturally this is the one receiving all of the attention, which shows how much I know :D

​I also don't think this one is attractive to me, Lambda is especially powerful when it's used together with other AWS services(S3, DynamoDB, Kinesis Streams, etc).
​​

I definitely don't think we should try to reimplement this from scratch in OpenStack. IMHO if we're going to add FaaS capabilities we should re-use some existing project (like OpenWhisk), even if we have to write our own native API over the top of it.

The things we'd really want it to do would be:

* Authenticate against Keystone,
* Provide Keystone credentials for the user-supplied functions it runs to access (probably using Keystone trusts), and
* Connect to existing OpenStack sources of events, which hopefully means Zaqar queues

Which sounds challenging to integrate with an existing standalone project, though still not as bad as writing an equivalent from scratch.

TBH I think the appeal, at least for the FaaS-as-a-PaaS (aka #serverless) crowd, is going to be pretty limited until such time as we have an equivalent of DynamoDB in OpenStack. (i.e. no time soon, since the MagnetoDB project is goneburger.) The idea of FaaS is to make the unit of compute power that you're paying for (a) as fine-grained as possible, and (b) scalable to infinity. Swift provides the same thing for storage (Nova:FaaS::Cinder:Swift). What we don't have is the equivalent for a database, there's only Trove where you're paying for a VM-sized chunk at a minimum and scaling up in units of VM-sized chunks until you reach the limit of how many VMs can communicate with each other and still get any work done. Not many web apps can get by without a database, so that largely negates the purpose to my mind, since the database will likely both dominate costs at the low end and put the upper limit on scale at the high end.

​Yeah, I agree with you that more things are needed so that FaaS-like stuff could be used appropriately and ideally, we can't get everything ready on day 1, that's how we do things,  from simple to complex, isn't it?



cheers,
Zane.

[1] https://www.openstack.org/videos/video/building-self-healing-applications-with-aodh-zaqar-and-mistral



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161103/de97eabc/attachment-0001.html>