<div dir="ltr"><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Nov 25, 2013 at 3:56 PM, Devananda van der Veen <span dir="ltr"><<a href="mailto:devananda.vdv@gmail.com" target="_blank">devananda.vdv@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra">Hi!</div><div class="gmail_extra"><br></div><div class="gmail_extra">Very good questions. I think most of them are directed towards the Ceilometer team, but I have answered a few bits inline.</div>
<div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote"><div class="im">On Mon, Nov 25, 2013 at 7:24 AM, wanghaomeng <span dir="ltr"><<a href="mailto:wanghaomeng@163.com" target="_blank">wanghaomeng@163.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;font-size:14px;font-family:arial"><div><br></div>Hello all:<div><br><div><div>Basically, I understand the solution is - <span style="line-height:1.7">Our Ironic will implement an IPMI driver</span></div>
</div></div></div></blockquote><div><br></div></div><div>We will need to add a new interface -- for example, ironic.drivers.base.BaseDriver:sensor and the corresponding ironic.drivers.base.SensorInterface class, then implement this interface as ironic.drivers.modules.ipmitool:IPMISensor</div>
<div><br></div><div>We also need to define the methods this interface supports and what the return data type is for each method. I imagine it may be something like:</div><div>- SensorInterface.list_available_sensors(node) returns a list of sensor names for that node</div>
<div>- SensorInterface.get_measurements(node, list_of_sensor_names) returns a dict of dicts, eg, { 'sensor_1': {'key': 'value'}, 'sensor_2': ...}</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div><span style="line-height:1.7">(extendable framework for more drivers) to collect hardware sensor data(cpu temp, fan speed, volts, etc) via IPMI protocol from hardware server node, and emit the </span><span style="line-height:1.7">AMQP </span><span style="line-height:1.7">message to Ceilometer Collector</span><span style="line-height:1.7">, Ceilometer have the framework to handle the valid sample message and save to the database for data retrieving by consumer.</span></div>
<div><br></div><div><span style="line-height:1.7">Now, how do you think if we should clearly define the </span><b style="line-height:1.7">interface & data model </b><span style="line-height:1.7">specifications</span><span style="line-height:1.7"> between Ironic and Ceilometer to enable IPMI data collecting, then our two team can start the coding together?</span></div>
</div></div></div></blockquote><div><br></div></div><div>I think this is just a matter of understanding Ceilometer's API so that Ironic can emit messages in the correct format. You've got many good questions for the Ceilometer team on this below.</div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div><br></div><div>And I still have some concern with our interface and data model as below, the spec need to be discussed and finalized:</div>
<div><br></div><div>1. What is the Ceilometer sample data mandatory attributes, such as instance_id/tenant_id/user_id/resource_id, if they are not optional, where are these data populated, from Ironic or Ceilomter side?<span style="font-size:small;line-height:normal;color:rgb(34,34,34)"> </span></div>
</div></div></div></blockquote></div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;font-size:14px;font-family:arial"><div><div>
<div><div> </div><div> <b><i>name/type/unit/volume/<span style="line-height:1.7">timestamp</span></i></b><span style="line-height:1.7"> - basic sample property, can be populated from Ironic side as data source</span></div>
<div> <b><i>user_id/project_id/resource_id</i></b> - Ironic or Ceilometer populate these fields??<span style="line-height:1.7"> </span></div></div></div></div></div></blockquote></div></div></div></div></blockquote><div>
<br></div><div><div class="gmail_default" style="font-size:small">Ceilometer knows nothing about resources unless it is told, so all of the required fields have to be provided by the sender.</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;font-size:14px;font-family:arial">
<div><div><div><div> <i><b>resource_metadata - </b>this is for Ceilometer metadata query, Ironic know nothing for such resource metadata I think</i></div></div></div></div></div></blockquote></div></div></div></div></blockquote>
<div><br></div><div><div class="gmail_default" style="font-size:small">The resource metadata depends on the resource type, but should be all of the user-visible attributes for that object stored in the database at the time the measurement is taken. For example, for instances we (try to) get all of the instance attributes.</div>
</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div>
<div> <i><b>source </b></i>- can we hard-code as 'hardware' as a source identifier?</div></div></div></div></div></blockquote></div></div></div></div></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">
No, the source is the source of the user and project ids, not the source of the measurement (the data source is implied by the meter name). The default source for user and project is "openstack" to differentiate from an add-on layer like a PaaS where there are different user or project ids.</div>
<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div><br></div></div></div></div></blockquote><div><br></div></div><div>Ironic can cache the user_id and project_id of the instance. These will not be present for unprovisioned nodes.</div>
<div><br></div><div>I'm not sure what "resource_id" is in this context, perhaps the nova instance_uuid? If so, Ironic has that as well.</div></div></div></div></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">
Do end-users know about bare metal servers before they are provisioned as instances? Can a regular user, for example, as for the list of available servers or find details about one by name or id?</div><br></div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div></div><div>2. Not sure if our Ceilometer only accept the <b>signed-message</b>, if it is case, how Ironic get the message trust for Ceilometer, and send the valid message which can be accepted by Ceilometer Collector?</div>
</div></div></div></blockquote></div></div></div></div></blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">I'm not sure it's appropriate for ironic to be sending messages using ceilometer's sample format. We receive data from the other projects using the more generic notification system, and that seems like the right tool to use here, too. Unless the other ceilometer devs disagree?</div>
<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div>
<div><br></div><div>3. What is the Ceilometer sample data structure, and what is the min data item set for the IPMI message <span style="line-height:1.7">be emitted to Collector?</span></div><div><div> <b>name/type/unit/volume/</b><span style="line-height:1.7"><i><span style="line-height:1.7"><b>timestamp/source</b> - is this min data item set?</span></i></span></div>
<div> </div></div><div>3. If the detailed data model should be defined for our IPMI data now?, what is our the first version scope, how many IPMI data type we should support? Here is a IPMI data sample list, I think we can support these as a min set.</div>
<div> <i><b>Temperature </b>- <span style="line-height:1.7">System Temp/</span><span style="line-height:1.7">CPU Temp</span></i></div><div><div><i> <b>FAN </b><span style="line-height:1.7"><b>Speed in rpm</b> - FAN 1/2/3/4/A</span></i></div>
<div><i> <b>Volts </b>- <span style="line-height:1.7">Vcore/</span><span style="line-height:1.7">3.3VCC/</span><span style="line-height:1.7">12V/</span><span style="line-height:1.7">VDIMM/</span><span style="line-height:1.7">5VCC/</span><span style="line-height:1.7">-12V/</span><span style="line-height:1.7">VBAT/</span><span style="line-height:1.7">VSB/</span><span style="line-height:1.7">AVCC</span></i></div>
</div></div></div></div></blockquote><div><br></div></div><div>I think that's a good starting list. We can add more later.</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div><br></div><div>4. More specs - such as naming conversions, common constant reference definitions ...</div></div></div></div></blockquote></div>
</div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial"><div><div><div><br></div><div>These are just a draft, not the spec, correct me if I am wrong understanding and add the missing aspects, we can discuss these interface and data model clearly I think.</div>
<div><br></div><div><br></div><div>----------------------------------------------------------</div><div><i>Haomeng</i></div><div><i>Thanks:)</i></div><div><div><div><br></div><div><br></div></div></div></div></div>
</div></blockquote><div><br></div></div><div>Cheers,</div><div>Devananda</div><div><div class="h5"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="line-height:1.7;font-size:14px;font-family:arial">
<div><div><div><div><div></div><div><div></div><div></div><br>At 2013-11-21 16:08:00,"Ladislav Smola" <<a href="mailto:lsmola@redhat.com" target="_blank">lsmola@redhat.com</a>> wrote:<br> <blockquote style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid">
<div>Responses inline.<br>
<br>
On 11/20/2013 07:14 PM, Devananda van der Veen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Responses inline.
<div><br>
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Nov 20, 2013 at 2:19 AM,
Ladislav Smola <span dir="ltr"><<a href="mailto:lsmola@redhat.com" target="_blank">lsmola@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Ok,
I'll try to summarize what will be done in the near
future for Undercloud monitoring.<br>
<br>
1. There will be Central agent running on the same
host(hosts once the central agent horizontal scaling is
finished) as Ironic<br>
</blockquote>
<div><br>
</div>
<div>Ironic is meant to be run with >1 conductor
service. By i-2 milestone we should be able to do this,
and running at least 2 conductors will be recommended.
When will Ceilometer be able to run with multiple
agents?</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Here it is described and tracked: <a href="https://blueprints.launchpad.net/ceilometer/+spec/central-agent-improvement" target="_blank">https://blueprints.launchpad.net/ceilometer/+spec/central-agent-improvement</a><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>On a side note, it is a bit confusing to call
something a "central agent" if it is meant to be
horizontally scaled. The ironic-conductor service has
been designed to scale out in a similar way to
nova-conductor; that is, there may be many of them in an
AZ. I'm not sure that there is a need for Ceilometer's
agent to scale in exactly a 1:1 relationship with
ironic-conductor?</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Yeah we have already talked about that. Maybe some renaming will be
in place later. :-) I don't think it has to be 1:1 mapping. There
was only requirement to have "Hardware agent" only on hosts with
ironic-conductor, so it has access to management network, right?<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
2. It will have SNMP pollster, SNMP pollster will be
able to get list of hosts and their IPs from Nova (last
time I<br>
checked it was in Nova) so it can poll them for
stats. Hosts to poll can be also defined statically in
config file.<br>
</blockquote>
<div><br>
</div>
<div>Assuming all the undercloud images have an SNMP
daemon baked in, which they should, then this is fine.
And yes, Nova can give you the IP addresses for
instances provisioned via Ironic.</div>
<div> </div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Yes.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
3. It will have IPMI pollster, that will poll Ironic
API, getting list of hosts and a fixed set of stats
(basically everything<br>
that we can get :-))<br>
</blockquote>
<div><br>
</div>
<div>No -- I thought we just agreed that Ironic will not
expose an API for IPMI data. You can poll Nova to get a
list of instances (that are on bare metal) and you can
poll Ironic to get a list of nodes (either nodes that
have an instance associated, or nodes that are
unprovisioned) but this will only give you basic
information about the node (such as the MAC addresses of
its network ports, and whether it is on/off, etc).</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Ok sorry I have misunderstood the:<br>
"If there is a fixed set of information (eg, temp, fan speed, etc)
that ceilometer will want,let's make a list of that and add a driver
interface within Ironic to abstract the collection of that
information from physical nodes. Then, each driver will be able to
implement it as necessary for that vendor. Eg., an iLO driver may
poll its nodes differently than a generic IPMI driver, but the
resulting data exported to Ceilometer should have the same
structure."<br>
<br>
I thought I've read the data will be exposed, but it will be just
internal Ironic abstraction, that will be polled by Ironic and send
directly do Ceilometer collector. So same as the point 4., right?
Yeah I guess this will be easier to implement.<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
4. Ironic will also emit messages (basically all events
regarding the hardware) and send them directly to
Ceilometer collector<br>
</blockquote>
<div><br>
</div>
<div>Correct. I've updated the BP:</div>
<div><br>
</div>
<div>
<a href="https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent" target="_blank">https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent</a><br>
</div>
<div><br>
</div>
<div>Let me know if that looks like a good description.</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
Yeah, seems great. I would maybe remove the word 'Agent', seems
Ironic will send it directly to Ceilometer collector, so Ironic acts
as agent, right?<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><br>
-Devananda<br>
</div>
<div><br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"></blockquote>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Does it seems to be correct? I think that is the basic
we must have to have Undercloud monitored. We can then
build on that.<br>
<br>
Kind regards,<br>
Ladislav
<div>
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div><br>
On 11/20/2013 09:22 AM, Julien Danjou wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">On
Tue, Nov 19 2013, Devananda van der Veen wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">If
there is a fixed set of information (eg, temp,
fan speed, etc) that<br>
ceilometer will want,<br>
</blockquote>
Sure, we want everything.<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">let's
make a list of that and add a driver interface<br>
within Ironic to abstract the collection of that
information from physical<br>
nodes. Then, each driver will be able to
implement it as necessary for that<br>
vendor. Eg., an iLO driver may poll its nodes
differently than a generic<br>
IPMI driver, but the resulting data exported to
Ceilometer should have the<br>
same structure.<br>
</blockquote>
I like the idea.<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">An
SNMP agent doesn't fit within the scope of
Ironic, as far as I see, so<br>
this would need to be implemented by Ceilometer.<br>
</blockquote>
We're working on adding pollster for that indeed.<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">As
far as where the SNMP agent would need to run,
it should be on the<br>
same host(s) as ironic-conductor so that it has
access to the<br>
management network (the physically-separate
network for hardware<br>
management, IPMI, etc). We should keep the
number of applications with<br>
direct access to that network to a minimum,
however, so a thin agent<br>
that collects and forwards the SNMP data to the
central agent would be<br>
preferable, in my opinion.<br>
</blockquote>
We can keep things simple by having the agent only
doing that polling I<br>
think. Building a new agent sounds like it will
complicate deployment<br>
again.<br>
<br>
</blockquote>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</blockquote></div></div></div></div></div></div><br><br><span title="neteasefooter"><span></span></span></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div>