Open Stack

On 03/05/2013 08:49 AM, Sandy Walsh wrote:
> 
> 
> On 03/05/2013 01:10 AM, Joshua Harlow wrote:
>> Hi all,
>>
>> I was just looking through the grizzly code and came upon a new goodie:
>>
>> /compute_utils.EventReporter/
>>
>> Is there a blueprint for how this is going to be used, it looks like its
>> being used to save 'states' for later 'resuming' for an orchestration
>> unit (conductor?)?
>>
>> Is that correct? I haven't been following this to much so was wondering
>> as yahoo! and nttdata 'centralized' orchestration unit prototype (aka
>> https://etherpad.openstack.org/the-future-of-orch) also needs such state
>> transition data, but we would like to keep usage of said states and
>> recording of states starting/finishing and such in 1 place (and not
>> dispersed throughout the code). I also see stuff
>> like https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L247 which
>> is creating records based off of function names, is there a reason we
>> can't actually solidify on what the different states are beforehand and
>> not creating them on the fly. Isn't that a more sane and longer-term
>> approach where said states and there transitions are in 1 place and not
>> all over?
>>
>> Just a personal concern but has there been any any determination of how
>> writing all this data into the DB (via the MQ) will affect DB size, its
>> a pretty big change from writing nothing in the DB (or vm_state and
>> task_state) to writing detailed information about when each function
>> starts and stops, has there been any size estimation on the change so
>> that deployers can ensure they adjust correctly? Maybe in general, but
>> is there any good design docs on the conductor, its features in grizzly
>> and such. If people are going to use it, I'd like to know what all its
>> doing in grizzly and what new database tables it adds, what new MQ
>> dependencies it adds, and so on. Does anyone else think that would be
>> useful?
>>
>> -Josh
> 
> Josh, thanks for writing this. I've been meaning to do something similar
> myself. We clearly have an issue with the myriad of ways notification
> events are being handled in Nova.
> 
> We have:
> 1. The AMQP notifications,
> 2. The DB InstanceFault table,
> 3. Exception handling decorators,
> 4. The conductor mechanisms you've highlighted,
> 5. and, to a lesser extent, logging
> 
> Recently I was asked about what it would take to get some of the data
> stored in the InstanceFaults into the notifications. (why do it twice?)
> Then there's the problem of squashing a private data leakage problem in
> one place to have it appear in some other place.
> 
> We are doing work on the Ceilometer side to shore up #1 to provide a
> meaningful data package, but there is clearly duplicated effort.
> 
> Personally I think storing ever-growing InstanceFault data in the
> production database is a bad idea. Our rule should be: "Get archival
> data away from production Nova as quickly as possible."
> 
> Sounds like we need a summit session to brainstorm ideas on unifying
> these things (logging excluded, perhaps)?

Just registered:
https://blueprints.launchpad.net/nova/+spec/unified-event-notifications

and put in a session proposal: http://summit.openstack.org/

-S

> -S
> 
> 
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>