Open Stack

On 11/01/2013 01:39 PM, Clayton Coleman wrote:
>
>
> ----- Original Message -----
>> On 11/01/2013 12:33 PM, Clayton Coleman wrote:
>>> ----- Original Message -----
>>>>
>>>> Once all the gitreview stuff is cleaned up I was going to do some purely
>>>> mechanical additions.
>>>>
>>>> I heard a few +1 for sqlalchemy with the standard OpenStack abstraction:
>>>>
>>>> solum/db/api.py
>>>>     manager abstraction for db calls
>>>> solum/db/sqlalchemy/api.py
>>>>     sqlalchemy implementation
>>>>
>>>> I was also going to throw in migrate as a dependency and put in the glue
>>>> code
>>>> for that based on common use from ironic/trove/heat.  That'll pull in a
>>>> few
>>>> openstack common and config settings.  Finally, was going to add a
>>>> solum-dbsync command a la the aforementioned projects.  No schema will be
>>>> added.
>>>>
>>>> Objections?
>>>>
>>>
>>> I was blindly assuming we want to pull in eventlet support, with the
>>> implicit understanding that we will be doing some form of timeslicing and
>>> async io bound waiting in the API... but would like to hear others weigh
>>> in before I add the monkey_patch and stub code around script startup.
>>
>> I'm not so sure that bringing in eventlet should be done by default. It
>> adds complexity and if most/all of the API calls will be doing some call
>> to a native C library like libmysql that blocks, I'm not sure there is
>> going to be much benefit to using eventlet versus multiplexing the
>> servers using full OS processes -- either manually like some of the
>> projects do with the workers=N configuration and forking, or using more
>> traditional multiplexing solutions like running many mod_wsgi or uwsgi
>> workers inside Apache or nginx.
>>
>
> What about callouts to heat/keystone APIs?

Sure, it's possible to do that with eventlet. It's also possible to do 
that with a queueing system. For example, the API server could send an 
RPC message to a queue and a separate process could work on executing 
the individual tasks associated with a particular application build-pack.

The design of Zuul [1], especially with the relatively recent addition 
of Gearman and the nodepool orchestrator [2] that the openstack-infra 
folks wrote would, IMO, be a worthy place to look for inspiration, since 
Solum essentially is handling a similar problem domain -- distributed 
processing of ordered tasks.

Best,
-jay

[1] https://github.com/openstack-infra/zuul
[2] https://github.com/openstack-infra/nodepool