<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
John, Don't forget two huge additional benefits to it being
external: Scalability and distribution. The service handling this
can be clustered and distributed if it is external. <br>
<br>
<br>
Aimon<br>
<br>
<br>
On 2/7/11 3:12 PM, John Purrier wrote:
<blockquote cite="mid:005101cbc71c$87058a10$95109e30$@org"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:black;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:662778216;
mso-list-type:hybrid;
mso-list-template-ids:-721052 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoPlainText">There are a lot of benefits to having an
external system be responsible for handling usage data from
Nova, Swift, or other OpenStack services. I would call out:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">1.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Simplification
of code within each service. The collection and publication of
usage data is “dumb”… store the service usage data in a
service defined schema (could just be logfiles) until it is
fetched/pushed and then delete. No additional service API’s
required beyond that to efficiently move the data out of the
service.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">2.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Simplification
of hardware required for each service. If the service is
required to do analytics/processing on the usage data this
will drive more powerful hardware solutions.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">3.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Single
target for organizational usage data. This allows the best
selection of hardware and software for collecting and
maintaining large datasets. This also allows the
analytic/billing/auditing/warehousing service to scale
independently of any particular service.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">4.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Data
warehousing for compliance and billing repeatability. It is a
likely scenario that billing and usage disputes will occur,
sometimes months after the fact. It is imperative that the
billing and compliance reports be able to be recreated and
analyzed. Holding all of this data inside the individual
services for long retention periods doesn’t make sense.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">5.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Data
warehousing for cold storage. The ability to tier storage for
long term retention, at the cheapest possible cost.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">6.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Being
able to do joins effectively across different service usages
to create consolidated billing.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">7.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->In
addition to billing and auditing use cases, being able to make
the data available to scheduled and ad hoc reporting. This
involved analytic processing, data reduction, and other
processing that is not appropriate on the provisioning and
real-time service controllers.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText" style="margin-left: 0.5in; text-indent:
-0.25in;"><!--[if !supportLists]--><span style="">8.<span
style="font: 7pt "Times New Roman";"> </span></span><!--[endif]-->Specialized
data processing to allow real-time updates to system
behaviors. This is getting into the future, but creating a
feed forward analytic process may allow some really smart
auto-scaling and dynamic provisioning scenarios.<o:p></o:p></p>
<p class="MsoListParagraph"><o:p> </o:p></p>
<p class="MsoPlainText">John<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">-----Original Message-----<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:openstack-bounces+john=openstack.org@lists.launchpad.net">openstack-bounces+john=openstack.org@lists.launchpad.net</a>
[<a class="moz-txt-link-freetext" href="mailto:openstack-bounces+john=openstack.org@lists.launchpad.net">mailto:openstack-bounces+john=openstack.org@lists.launchpad.net</a>]
On Behalf Of Monsyne Dragon<br>
Sent: Monday, February 07, 2011 2:58 PM<br>
To: <a class="moz-txt-link-abbreviated" href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Subject: Re: [Openstack] Pondering multi-tenant needs in nova.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">On 2/7/11 2:49 PM, Eric Day wrote:<o:p></o:p></p>
<p class="MsoPlainText">> Thanks for explaining things
further, Jay.<o:p></o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">> I agree if we want external systems
poking into Nova for audit/billing<o:p></o:p></p>
<p class="MsoPlainText">> queries, then yes, this gets
inefficient. My assumption is that Nova<o:p></o:p></p>
<p class="MsoPlainText">> specific DBs only contain
operational data required for production and<o:p></o:p></p>
<p class="MsoPlainText">> it would push billing/audit events
to some external system that can<o:p></o:p></p>
<p class="MsoPlainText">> collect, aggregate, and answer
those queries efficiently. Trying to<o:p></o:p></p>
<p class="MsoPlainText">> design a common data store that
fits both use cases of provisioning<o:p></o:p></p>
<p class="MsoPlainText">> instances/networks/volumes along
with handling queries for<o:p></o:p></p>
<p class="MsoPlainText">> billing/audit would be difficult
(as we are seeing). Pushing<o:p></o:p></p>
<p class="MsoPlainText">> billing/audit data to another
system gives us the flexibility to<o:p></o:p></p>
<p class="MsoPlainText">> choose the most suitable data store
and querying abilities for each<o:p></o:p></p>
<p class="MsoPlainText">> use case without making sacrifices
for the other.<o:p></o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText"><span style="color: black;"><o:p> </o:p></span></p>
<p class="MsoPlainText">Yes this is the model proposed with the
system-usage blueprint. Nova <o:p></o:p></p>
<p class="MsoPlainText">publishes usage data, via a pub/sub
interface, and a separate <o:p></o:p></p>
<p class="MsoPlainText">billing/audit system subscribes to those
events, and builds it's <o:p></o:p></p>
<p class="MsoPlainText">datastore as it sees fit.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
</div>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
Mailing list: <a class="moz-txt-link-freetext" href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a>
Post to : <a class="moz-txt-link-abbreviated" href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a>
Unsubscribe : <a class="moz-txt-link-freetext" href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a>
More help : <a class="moz-txt-link-freetext" href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Aimon Bustardo | Morph Labs | +1.310.625.0608 (mobile) | +1.310.437.4898 (office) | <a class="moz-txt-link-abbreviated" href="http://www.morphlabs.com">www.morphlabs.com</a> | <a class="moz-txt-link-abbreviated" href="mailto:aimon@mor.ph">aimon@mor.ph</a>
</pre>
</body>
</html>