Open Stack

On Mon, Oct 20, 2014 at 11:12:57AM -0700, Dan Smith wrote:
> > OK, so in reviewing Dan B's patch series that refactors the virt
> > driver's get_available_resource() method [1], I am stuck between two
> > concerns. I like (love even) much of the refactoring work involved in
> > Dan's patches. They replace a whole bunch of our nested dicts that are
> > used in the resource tracker with real objects -- and this is something
> > I've been harping on for months that really hinders developer's
> > understanding of Nova's internals.
> 
> dict['line1'] = 'Agreed, this is extremely important stuff.'
> dict['line2'] = 'The current dict mess that we have there is '
> dict['line3'] = 'really obscure and confusing.'
> reply = jsonutils.dumps(dict)
> 
> > However, all of the object classes that Dan B has introduced have been
> > unversioned objects -- i.e. they have not derived from
> > nova.objects.base.NovaObject. This means that these objects cannot be
> > sent over the wire via an RPC API call. In practical terms, this issue
> > has not yet reared its head, because the resource tracker still sends a
> > dictified JSON representation of the object's fields directly over the
> > wire, in the same format as Icehouse, therefore there have been no
> > breakages in RPC API compatibility.
> 
> Right, so the blueprint for this work states that it's not to be sent
> over the RPC wire or stored in the database. However, it already is in
> some cases (at least the ComputeNode object has the unversioned
> JSONified version of some of these hardware models in it).
> 
> If the modeling is purely for internal-to-compute-node purposes, then
> it's all good. However, it surely seems like with the pending scheduler
> isolation work, we're in a spot where we are building two parallel model
> hierarchies, and I'm not really sure why.

The rationale behind two parallel data model hiercharies is that the
format the virt drivers report data in, is not likely to be exactly
the same as the format that the resoure tracker / scheduler wishes to
use in the database.

If we have a single hierarchy, then  whenever we need to update the
data format for the schedular to improve its performance or flexibility
then we have a ripple effect where we'd have to update all the virt driver
implementations too.

Based on what I've seen about the extensible resource tracker it seems
like over time there was going to be greater divergance between what
the virt drivers support and how the ERT  wants to persist it after
transforming it into a easier to deal with structure.

> > My proposal is that before we go and approve any BPs or patches that add
> > to nova/virt/hardware.py, we first put together a patch series that
> > moves the object models in nova/virt/hardware.py to being full-fledged
> > objects in nova/objects/*
> 
> I'm not sure that just converting them all to NovaObjects is really
> necessary here. If it's all stuff that is going to go over the wire
> eventually as part of the resource tracker's expansion, then probably
> so. If there are bits of the model that only serve to let the resource
> tracker do its calculations, then perhaps it doesn't make sense to
> require those be NovaObjects.

Yep, pretty much agree with that position. What's difficult is that
we're in a bit of a transition stage where stuff that's reported now
does get stuffed straight into the database, but in the future might
well undergo translation/calculation prior to being stuff in the
database.

If we convert everything into NovaObjects based on what is directly
stored in the DB today, long term this might have proved to be
uneccessary.  So I took the pragmatic decision that I'd define some
plain Python classes, rather than NovaObject clases on the basis that
it was still an improvement over an undocumented dict. And as & when
the scheduler/resource tracker refactor settled down we could either
stick with the plain python classes, or convert them into full Nova
objects as applicable.

> Regardless, it sounds like we need some discussion on how best to
> proceed here. Since it's entirely wrapped up in the scheduler work, we
> should definitely try to make sure that what we're doing here fits with
> those plans. Last I heard, we weren't sure where we were going to draw
> the line between nova bits and scheduler bits, so erring on the side of
> "more versioned interfaces" seems safest to me.


FWIW, my patch series is logically split up into two parts. THe first
10 or so patches are just thought of as general cleanup and useful to
Nova regardless of what we decide todo. The second 10 or so patches
are where the objects start appearing & getting used & the controversial
bits needing mor detailed discussion.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|