<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:black;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:662778216;
mso-list-type:hybrid;
mso-list-template-ids:-721052 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoPlainText>There are a lot of benefits to having an external system be responsible for handling usage data from Nova, Swift, or other OpenStack services. I would call out:<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Simplification of code within each service. The collection and publication of usage data is “dumb”… store the service usage data in a service defined schema (could just be logfiles) until it is fetched/pushed and then delete. No additional service API’s required beyond that to efficiently move the data out of the service.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Simplification of hardware required for each service. If the service is required to do analytics/processing on the usage data this will drive more powerful hardware solutions.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Single target for organizational usage data. This allows the best selection of hardware and software for collecting and maintaining large datasets. This also allows the analytic/billing/auditing/warehousing service to scale independently of any particular service.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>4.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Data warehousing for compliance and billing repeatability. It is a likely scenario that billing and usage disputes will occur, sometimes months after the fact. It is imperative that the billing and compliance reports be able to be recreated and analyzed. Holding all of this data inside the individual services for long retention periods doesn’t make sense.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>5.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Data warehousing for cold storage. The ability to tier storage for long term retention, at the cheapest possible cost.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>6.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Being able to do joins effectively across different service usages to create consolidated billing.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>7.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>In addition to billing and auditing use cases, being able to make the data available to scheduled and ad hoc reporting. This involved analytic processing, data reduction, and other processing that is not appropriate on the provisioning and real-time service controllers.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>8.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Specialized data processing to allow real-time updates to system behaviors. This is getting into the future, but creating a feed forward analytic process may allow some really smart auto-scaling and dynamic provisioning scenarios.<o:p></o:p></p><p class=MsoListParagraph><o:p> </o:p></p><p class=MsoPlainText>John<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>-----Original Message-----<br>From: openstack-bounces+john=openstack.org@lists.launchpad.net [mailto:openstack-bounces+john=openstack.org@lists.launchpad.net] On Behalf Of Monsyne Dragon<br>Sent: Monday, February 07, 2011 2:58 PM<br>To: openstack@lists.launchpad.net<br>Subject: Re: [Openstack] Pondering multi-tenant needs in nova.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>On 2/7/11 2:49 PM, Eric Day wrote:<o:p></o:p></p><p class=MsoPlainText>> Thanks for explaining things further, Jay.<o:p></o:p></p><p class=MsoPlainText>><o:p> </o:p></p><p class=MsoPlainText>> I agree if we want external systems poking into Nova for audit/billing<o:p></o:p></p><p class=MsoPlainText>> queries, then yes, this gets inefficient. My assumption is that Nova<o:p></o:p></p><p class=MsoPlainText>> specific DBs only contain operational data required for production and<o:p></o:p></p><p class=MsoPlainText>> it would push billing/audit events to some external system that can<o:p></o:p></p><p class=MsoPlainText>> collect, aggregate, and answer those queries efficiently. Trying to<o:p></o:p></p><p class=MsoPlainText>> design a common data store that fits both use cases of provisioning<o:p></o:p></p><p class=MsoPlainText>> instances/networks/volumes along with handling queries for<o:p></o:p></p><p class=MsoPlainText>> billing/audit would be difficult (as we are seeing). Pushing<o:p></o:p></p><p class=MsoPlainText>> billing/audit data to another system gives us the flexibility to<o:p></o:p></p><p class=MsoPlainText>> choose the most suitable data store and querying abilities for each<o:p></o:p></p><p class=MsoPlainText>> use case without making sacrifices for the other.<o:p></o:p></p><p class=MsoPlainText>><o:p> </o:p></p><p class=MsoPlainText><span style='color:black'><o:p> </o:p></span></p><p class=MsoPlainText>Yes this is the model proposed with the system-usage blueprint. Nova <o:p></o:p></p><p class=MsoPlainText>publishes usage data, via a pub/sub interface, and a separate <o:p></o:p></p><p class=MsoPlainText>billing/audit system subscribes to those events, and builds it's <o:p></o:p></p><p class=MsoPlainText>datastore as it sees fit.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText><o:p> </o:p></p></div></body></html>