<div dir="ltr"><div><font>Hello Dmitriy!<br><br>Of course event_id could identify row. Actually Row structure now is a question for discuss.<br>The main feature of HBase that data is stored in sorted rows.<br><br></font><font>To filter events by time interval we will need to scan all stored data, when <br>
rowkey is started with event_id. </font><font>Instead of adding timestamp in rowkey, we could set <br>timestamps as a additional column value or qualifier inside row.<br><br>Another decision is to start rowkey with "timestamp + event_id".<br>
In this case we will have events stored by </font><font>timestamps </font><font>in HBase.<br></font><br></div><div><font>What type of rowkey will give more efficiency is a question.</font><br></div><font></font><div><div>
<div><div><div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 30, 2014 at 4:36 PM, Dmitriy Ukhlov <span dir="ltr"><<a href="mailto:dukhlov@mirantis.com" target="_blank">dukhlov@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Hello Igor!<br><br></div>Could you clarify, please, Why do we need <span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">"event_id + </span><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">reversed_timestamp" row key?<br>
</span></div><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> Isn't </span><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">"event_id" identify row?<br>
</span></div><div> </div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">
On Tue, Apr 29, 2014 at 11:08 AM, Igor Degtiarov <span dir="ltr"><<a href="mailto:idegtiarov@mirantis.com" target="_blank">idegtiarov@mirantis.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="h5"><div dir="ltr"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">Hi, everybody.</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">I’ve started to work on implementation of Event in ceilometer on HBase backend in the edges of blueprint </span><a href="https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-feature" style="text-decoration:none" target="_blank"><span style="font-size:15px;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-feature</span></a><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> </span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">By now Events has been implemented only in SQL. </span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">You know, using SQL we can build any query we need.</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">With HBase it is another story. The data structure is built basing on queries we need, so </span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">to construct the structure of Event on HBase, it is very important to answer the question what queries should be implemented to retrieve events from storage.</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">I registered bp </span><a href="https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-structure" style="text-decoration:none" target="_blank"><span style="font-size:15px;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-structure</span></a><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> for discussion Events structure in HBase.</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">For today it is prepared preliminary structure of Events in HBase:</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">table: Events</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> - rowkey: event_id + </span><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">reversed_timestamp<br>
</span></p><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> </span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> - column: event_type => string with description of event</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal"> - [list of columns: trait_id + trait_desc + trait_type=> trait_data]</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">Structure that is proposed will support next queries:</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">- event’s generation time </span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">- event id</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">- event type</span></p>
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">- trait: id, description, type</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">Any thoughts about additional queries that are necessary for Events.</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">I’ll publish the patch with current implementation soon.</span></p>
<br><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">Sincerely,</span></p>
<span style="vertical-align:baseline;font-variant:normal;font-style:normal;font-size:15px;background-color:transparent;text-decoration:none;font-family:Arial;font-weight:normal">Igor Degtiarov</span></div>
<br></div></div>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><span class=""><font color="#888888"><br><br clear="all"><br>-- <br><div dir="ltr"><div><div>Best regards,<br></div>Dmitriy Ukhlov<br></div>Mirantis Inc.<br></div>
</font></span></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div></div></div></div></div></div></div>