[Openstack] Monitoring / Billing Architecture proposed

Sandy Walsh sandy.walsh at rackspace.com
Mon Apr 23 20:09:01 UTC 2012


Flavor information is copied to the Instance table on creation so the
Flavors can change and still be tracked in the Instance. It may just
need to be sent in the notification payload.

The current events in the system are documented here:
http://wiki.openstack.org/SystemUsageData

-Sandy


On 04/23/2012 02:50 PM, Brian Schott wrote:
> So, we could build on this. No reason to reinvent, but we might want to
> expand the number of events.  I'm concerned about things like what
> happens when flavors change over time.  Maybe the answer is, always
> append to the flavor/instance-type table.  The code I remember and the
> admin interface that Ken wrote allowed you to modify flavors.  That
> would break billing unless you also track flavor modifications.
> 
> -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott at nimbisservices.com <mailto:brian.schott at nimbisservices.com>
> ph: 443-274-6064  fx: 443-274-6060
> 
> 
> 
> On Apr 23, 2012, at 1:40 PM, Luis Gervaso wrote:
> 
>> I have been looking at : http://wiki.openstack.org/SystemUsageData
>>
>> On Mon, Apr 23, 2012 at 7:35 PM, Brian Schott
>> <brian.schott at nimbisservices.com
>> <mailto:brian.schott at nimbisservices.com>> wrote:
>>
>>     Is there a document somewhere on what events the services emit?  
>>
>>     -------------------------------------------------
>>     Brian Schott, CTO
>>     Nimbis Services, Inc.
>>     brian.schott at nimbisservices.com
>>     <mailto:brian.schott at nimbisservices.com>
>>     ph: 443-274-6064 <tel:443-274-6064>  fx: 443-274-6060
>>     <tel:443-274-6060>
>>
>>
>>
>>     On Apr 23, 2012, at 12:39 PM, Monsyne Dragon wrote:
>>
>>>     This already exists in trunk.  The Notification system was
>>>     designed specifically to feed billing and monitoring systems. 
>>>
>>>     Basically, we don't want Nova/Glance/etc to be in the business of
>>>     trying to determine billing logic, since it is different for
>>>     pretty much everyone,  so we just emit notifications to a queue
>>>     and the interested pull what they want, and aggregate according
>>>     to their own rules. 
>>>
>>>     On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote:
>>>
>>>>     Hi,
>>>>
>>>>     I want to share the architecture i am developing in order to
>>>>     perform the monitorig / billing OpenStack support:
>>>>
>>>>     1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>>>     interchangeable) (Own Stuff or ServiceMix / Camel)
>>>      
>>>>     2. Events should be stored on a NoSQL document oriented database
>>>>     (I think mongodb is perfect, since we can query in a super easy
>>>>     fashion)
>>>
>>>     We have an existing system called Yagi
>>>     (https://github.com/Cerberus98/yagi/) that listens to the
>>>     notification queues and persists events to a Redis database.  It
>>>     then provides feeds as ATOM formatted documents that a billing
>>>     system can pull to aggregate data, It also can support PubSub
>>>     notification of clients thru the pubsubhubub protocol, and push
>>>     events to a long-term archiving store thru the AtomPub protocol. 
>>>
>>>     That said, the notification system outputs its events as JSON, so
>>>     it should be easy to pipe into a json document-oriented db if
>>>     that's what you need. (we only use ATOM because we have a
>>>     atom-based archiving/search/aggregation engine (it's open
>>>     source: http://atomhopper.org/ ) our in-house systems already
>>>     plug into. )
>>>
>>>
>>>
>>>>
>>>>     3a. The monitoring system can pull/push MongoDB
>>>>
>>>>     3b. The billing system can pull to create invoices 
>>>>
>>>>     4. A mediation EIP should be necessary to integrate a
>>>>     billing/monitoring product. (ServiceMix / Camel)
>>>>
>>>>     This is to receive your feedback. So please, critics are welcome!
>>>>
>>>>     Cheers!
>>>>
>>>>     -- 
>>>>     -------------------------------------------
>>>>     Luis Alberto Gervaso Martin
>>>>     Woorea Solutions, S.L
>>>>     CEO & CTO
>>>>     mobile: (+34) 627983344 <tel:%28%2B34%29%20627983344>
>>>>     luis@ <mailto:luis.gervaso at gmail.com>woorea.es <http://woorea.es/>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     Mailing list: https://launchpad.net/~openstack
>>>>     Post to     : openstack at lists.launchpad.net
>>>>     <mailto:openstack at lists.launchpad.net>
>>>>     Unsubscribe : https://launchpad.net/~openstack
>>>>     More help   : https://help.launchpad.net/ListHelp
>>>
>>>     --
>>>     Monsyne M. Dragon
>>>     OpenStack/Nova 
>>>     cell 210-441-0965 <tel:210-441-0965>
>>>     work x 5014190
>>>
>>>     _______________________________________________
>>>     Mailing list: https://launchpad.net/~openstack
>>>     Post to     : openstack at lists.launchpad.net
>>>     <mailto:openstack at lists.launchpad.net>
>>>     Unsubscribe : https://launchpad.net/~openstack
>>>     More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> -- 
>> -------------------------------------------
>> Luis Alberto Gervaso Martin
>> Woorea Solutions, S.L
>> CEO & CTO
>> mobile: (+34) 627983344
>> luis@ <mailto:luis.gervaso at gmail.com>woorea.es <http://woorea.es/>
>>
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp




More information about the Openstack mailing list