Open Stack

On 01/11/16 22:20, Lingxian Kong wrote:
> Hi, all,
>
> Recently when I was talking with some customers of our OpenStack based
> public cloud, some of them are expecting to see a service similar to AWS
> Lambda in OpenStack ecosystem (so such service could be invoked by Heat,
> Mistral, Swift, etc.).

This is a really interesting space. There seems to be two main use cases 
for Lambda that are probably worth talking about separately:

The first is for just Lambda alone. You can use it to provide some glue 
logic between the other AWS services, so you can trigger off various 
events (e.g. S3 notifications) and write a little bit of conditioning 
logic that transforms the data and dispatches it to other services (e.g. 
DynamoDB). This one is particularly interesting to me, and in fact we 
can support parts of this in OpenStack already[1] because Mistral's 
functionality is equivalent to something like SWS + parts of Lambda. 
(Specifically, Mistral can do the data dispatch easily enough, but any 
data transformation has to be done in YAQL, which is a pretty high bar 
compared to just writing some code in a language of your choosing.)

The second one is Lambda + the API Gateway, which allows you to have web 
requests act as triggers, so that you can effectively treat it as a PaaS 
and build an entire web app by stringing together Lambda functions and 
the various other services (S3, DynamoDB, &c.). On the face of it this 
sounds to me like a gimmicky way of deploying an unmaintainable mess. 
Naturally this is the one receiving all of the attention, which shows 
how much I know :D

> Coincidently, I happened to see an introduction of OpenWhisk project by
> IBM guys in Barcelona Summit. The demo was great and I was much more
> exsited to know it was opensourced, but after checking, I feels a little
> bit frustrated, most of the core part of the code was written in Scala
> so it sets a high bar for me (yeah, I'm using Python) to learn and
> understand.
>
> So I came here to ask if there are people who are interested in
> serverless area as me or have the same requirements as our customers?
> Does it deserve a new project complies with OpenStack rules and
> conventions? Is there any chance that people could join together for the
> implementation?

I definitely don't think we should try to reimplement this from scratch 
in OpenStack. IMHO if we're going to add FaaS capabilities we should 
re-use some existing project (like OpenWhisk), even if we have to write 
our own native API over the top of it.

The things we'd really want it to do would be:

* Authenticate against Keystone,
* Provide Keystone credentials for the user-supplied functions it runs 
to access (probably using Keystone trusts), and
* Connect to existing OpenStack sources of events, which hopefully means 
Zaqar queues

Which sounds challenging to integrate with an existing standalone 
project, though still not as bad as writing an equivalent from scratch.

TBH I think the appeal, at least for the FaaS-as-a-PaaS (aka 
#serverless) crowd, is going to be pretty limited until such time as we 
have an equivalent of DynamoDB in OpenStack. (i.e. no time soon, since 
the MagnetoDB project is goneburger.) The idea of FaaS is to make the 
unit of compute power that you're paying for (a) as fine-grained as 
possible, and (b) scalable to infinity. Swift provides the same thing 
for storage (Nova:FaaS::Cinder:Swift). What we don't have is the 
equivalent for a database, there's only Trove where you're paying for a 
VM-sized chunk at a minimum and scaling up in units of VM-sized chunks 
until you reach the limit of how many VMs can communicate with each 
other and still get any work done. Not many web apps can get by without 
a database, so that largely negates the purpose to my mind, since the 
database will likely both dominate costs at the low end and put the 
upper limit on scale at the high end.

cheers,
Zane.

[1] 
https://www.openstack.org/videos/video/building-self-healing-applications-with-aodh-zaqar-and-mistral