Open Stack

> > > I really don't see much issue with ceilometer just doing
> > > those few things directly. The only private implementation
> > > assumption you'd be making about the Nova libvirt driver is
> > > that the libvirt VM name is the same as the instance name.
> > 
> > But just to confirm I understand the suggestion, by *direct* usage
> > do you mean simply calling the currently installed nova virt code
> > but at a level that is unlikely to change? (say by pushing some
> > internals of the driver implementation into the driver API)
> 
> Nope, don't use any code - just have your own code for talking
> directly to libvirt and other hypervisors.

OK. Granted we could do a greenfield re-write of the nova code
that talks to the hypervisors, but realistically the likelihood
is that ctrl-c/ctrl-v would provide the starting point. 

(So it smells a little like the "copy and distill" approach
mentioned earlier, even if tightly constrained in scope)

Seems to me this involves unnecessary duplication and leakage
of nova concerns into ceilo. If a bug is discovered on the nova
side, would we expect the nova dev doing the fix to go hunt in
the ceilo codebase to see if the fix needed to be ported there too?

Or is the thought that this code is so small (3% or whatever)
and thinly layered over the hypervisor, that its likely to have
had all the issues already smoked out?

> (And for clarity - this is if ceilometer must do the polling
> directly. I still think notifications are the way to go for now)

Understood.

> > > > There's been a fair amount of fuzziness around the
> > > > distinction between instrumentation and monitoring, but
> > > > this falls into the latter camp for me. Its not so much
> > > > about the internal dynamics of nova, as the user-visible
> > > > behavior (e.g. is my instance running hot or am I pushing
> > > > loads of data through the network, as opposed how long
> > > > did that API call to update the instance take to service
> > > > end-to-end or how many connections are in that pool).
> > > 
> > > Metering vs monitoring/instrumentation, not monitoring vs
> > > instrumentation :)
> > 
> > I kinda see the split more as metering/monitoring vs
> > instrumentation.
> 
> I don't see why.

Bacause:

(a) Metering and user-oriented monitoring are conceptually similar,
    in that both ask questions about the user-view of cloud resources.
    Metering asks about who's using these resources and for how long,
    monitoring asks how these resources are doing.

and:

(b) The data produced by metering and user-oriented monitoring
    are often derived from the same measurements, e.g. the CPU
    time exposed by libvirt would be reported by metering as the
    cumulative cpu_time meter, and by monitoring as the
    CPUUtilization metric. So there seems to be an opportunity
    here to avoid duplicating the measurement.

> Timeliness is a concern for user-oriented monitoring
> but not for metering.

Not as central a concern, but metering still requires somewhat of
a regular cadence.
 
> Timeliness is the issue in this discussion.

Yes ... and I see this as an implementation challenge, as opposed
to a reason for ceilo not to be concerned with the user-oriented
monitoring aspect.

> So, they're clearly not the same thing for the purposes of this
> discussion.

I didn't intend to present them as the *same thing* necessarily,
more to suggest that they share lots of commonality (certainly
at a conceptual level, also potentially in implementation). 
 
> > Well, actually more like metering/user-oriented-monitoring vs
> > cloud-operator-oriented-monitoring/instrumentation.
> > 
> > Currently ceilo is aiming to at least address both metering and
> > user-oriented-monitoring, and possibly more in time.
> > 
> > > If we were just designing a solution for metering, would we
> > > go for the notifications option?
> > 
> > Probably, but the scope of ceilo has widened a bit from pure
> > metering.
> 
> Right. From a far remove, that looks like a mistake at this early
> stage.

OK, well its a point we've gotten to after a lot of discussion
(at the summit and even prior to that) and the widening of scope
was also made clear at the TC meeting that endorsed ceilo for
incubation.

Are you advocating for (a) a strategic re-narrowing of scope, or
(b) a tactical decision to concentrate on just metering for now?

(Or something else?)
 
> As Sandy rightly says, if we just focused on metering, then the
> conclusion is simple - improve the notifications infrastructure. If
> Ceilometer focused on its original mission, forward progress would be
> easier.

OK, so this feels like avoiding a technical challenge by narrowing
the project scope. Which would be fine if the problem was really
nasty and intractable, but I'm not sure that's the case here.

Cheers,
Eoghan