<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">Hello
openstackers,</span>
<div class="moz-forward-container"> <br>
<span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span>
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">We
are working on MagnetoDB project and trying our best to follow
OpenStack standards.</span></p>
<br>
<span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span>
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">So,
MagnetoDB is aimed to be high performance scalable OpenStack
based WSGI application which provide interface to high
available distributed reliable key-value storage. We
investigated best practices and separated the next points:</span></p>
<ol style="margin-top:0pt;margin-bottom:0pt;">
<li dir="ltr"
style="list-style-type:decimal;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">to
avoid problems with GIL our application should be executed
in single thread mode with non-blocking IO (using
greenlets or another python specific approaches to rich
this)</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">to
make MagnetoDB scalable it is necessary to make MagnetoDB
stateless. It allows us run a lot of independent MagnetoDB
processes and switch all requests flow between them:</span></p>
</li>
<ol style="margin-top:0pt;margin-bottom:0pt;">
<li dir="ltr"
style="list-style-type:lower-alpha;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">at
single node to load all CPU’s cores</span></p>
</li>
<li dir="ltr"
style="list-style-type:lower-alpha;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">at
the different nodes for horizontal scalability</span></p>
</li>
</ol>
<li dir="ltr"
style="list-style-type:decimal;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">use
Cassandra as most reliable and mature distributed
key-value storage</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">use
datastax python-driver as most modern cassandra python
client which supports newest CQL3 and Cassandra native
binary protocol features set</span></p>
</li>
</ol>
<br>
<span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"></span>
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">So,
considering this points The next technologies was chosen:</span></p>
<ol style="margin-top:0pt;margin-bottom:0pt;">
<li dir="ltr"
style="list-style-type:decimal;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">gevent
as one of the fastest non-blocking single-thread WSGI
server. It is based on greenlet library and supports
monkey patching of standard threading library. It is
necessary because of datastax python driver uses threading
library and it’s backlog has task to add gevent backlog.
(We patched python-driver ourselves to enable this feature
as temporary solution and waiting for new python-driver
releases). It makes gevent more interesting to use than
other analogs (like eventlet for example)</span></p>
</li>
<li dir="ltr"
style="list-style-type:decimal;font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">gunicorn
as WSGI server which is able to run a few worker processes
and master process for workers managing and routing
request between them. Also it has integration with gevent
and can run gevent based workers. We also analyzed
analogues, such as uWSGI. It looks like more faster but
unfortunately we didn’t manage to work uWSGI in multi
process mode with MagnetoDB application.</span></p>
</li>
</ol>
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
Also I want to add that currently oslo wsgi framework is used
for organizing request routing. I know that current OpenStack
trend is to migrate WSGI services to Pecan wsgi framework.
Maybe is it reasonable for MagnetoDB too.<br>
</span></p>
<p dir="ltr"
style="line-height:1.15;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
We would like to hear your opinions about the libraries and
approaches we have chosen and would appreciate you help and
support in order to find the best balance between performance,
developer friendness and OpenStack standards.</span><br>
</p>
<pre class="moz-signature" cols="72">--
Best regards,
Dmitriy Ukhlov
Mirantis Inc.</pre>
</div>
</body>
</html>