<div dir="ltr"><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Mar 18, 2014 at 2:51 PM, Dmitriy Ukhlov <span dir="ltr"><<a href="mailto:dukhlov@mirantis.com" target="_blank">dukhlov@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">Hello
openstackers,</span>
<div> <br>
<span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"></span>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">We
are working on MagnetoDB project and trying our best to follow
OpenStack standards.</span></p>
<br>
<span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"></span>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">So,
MagnetoDB is aimed to be high performance scalable OpenStack
based WSGI application which provide interface to high
available distributed reliable key-value storage. We
investigated best practices and separated the next points:</span></p>
<ol style="margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">to
avoid problems with GIL our application should be executed
in single thread mode with non-blocking IO (using
greenlets or another python specific approaches to rich
this)</span></p>
</li>
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">to
make MagnetoDB scalable it is necessary to make MagnetoDB
stateless. It allows us run a lot of independent MagnetoDB
processes and switch all requests flow between them:</span></p>
</li>
<ol style="margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:lower-alpha;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">at
single node to load all CPU’s cores</span></p>
</li>
<li dir="ltr" style="vertical-align:baseline;list-style-type:lower-alpha;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">at
the different nodes for horizontal scalability</span></p>
</li>
</ol>
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">use
Cassandra as most reliable and mature distributed
key-value storage</span></p>
</li>
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">use
datastax python-driver as most modern cassandra python
client which supports newest CQL3 and Cassandra native
binary protocol features set</span></p>
</li>
</ol>
<br>
<span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"></span>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">So,
considering this points The next technologies was chosen:</span></p>
<ol style="margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">gevent
as one of the fastest non-blocking single-thread WSGI
server. It is based on greenlet library and supports
monkey patching of standard threading library. It is
necessary because of datastax python driver uses threading
library and it’s backlog has task to add gevent backlog.
(We patched python-driver ourselves to enable this feature
as temporary solution and waiting for new python-driver
releases). It makes gevent more interesting to use than
other analogs (like eventlet for example)</span></p>
</li>
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">gunicorn
as WSGI server which is able to run a few worker processes
and master process for workers managing and routing
request between them. Also it has integration with gevent
and can run gevent based workers. We also analyzed
analogues, such as uWSGI. It looks like more faster but
unfortunately we didn’t manage to work uWSGI in multi
process mode with MagnetoDB application.</span></p>
</li>
</ol>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"><br>
Also I want to add that currently oslo wsgi framework is used
for organizing request routing. I know that current OpenStack
trend is to migrate WSGI services to Pecan wsgi framework.
Maybe is it reasonable for MagnetoDB too.<br></span></p></div></div></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">The WSGI framework in Oslo is officially deprecated, and should no longer be used for new projects. Its use in existing projects is being phased out as projects create backwards-incompatible API changes that make moving to new tools safe (there's no point in creating issues by rewriting the old services). You should not use the old WSGI framework if you plan to propose MagenetoDB for incubation.</div>
<div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Several integrated and incubated projects have already adopted Pecan, so there are quite a few examples available to get you started. Drop by #pecanpy on freenode if you have questions.</div>
<div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Doug</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">
</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"><br>
We would like to hear your opinions about the libraries and
approaches we have chosen and would appreciate you help and
support in order to find the best balance between performance,
developer friendness and OpenStack standards.</span><span class="HOEnZb"><font color="#888888"><br>
</font></span></p><span class="HOEnZb"><font color="#888888">
<pre cols="72">--
Best regards,
Dmitriy Ukhlov
Mirantis Inc.</pre>
</font></span></div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>