Open Stack

Hello all:


Basically, I understand the solution is - Our Ironic will implement an IPMI driver(extendable framework for more drivers) to collect hardware sensor data(cpu temp, fan speed, volts, etc) via IPMI protocol from hardware server node, and emit the AMQP message to Ceilometer Collector, Ceilometer have the framework to handle the valid sample message and save to the database for data retrieving by consumer.


Now, how do you think if we should clearly define the interface & data model specifications between Ironic and Ceilometer to enable IPMI data collecting, then our two team can start the coding together?


And I still have some concern with our interface and data model as below, the spec need to be discussed and finalized:


1. What is the Ceilometer sample data mandatory attributes, such as instance_id/tenant_id/user_id/resource_id, if they are not  optional, where are these data populated, from Ironic or Ceilomter side?
 
  name/type/unit/volume/timestamp - basic sample property, can be populated from Ironic side as data source
  user_id/project_id/resource_id - Ironic or Ceilometer populate these fields??  
  resource_metadata - this is for Ceilometer metadata query, Ironic know nothing for such resource metadata I think
  source - can we hard-code as 'hardware' as a source identifier?


2. Not sure if our Ceilometer only accept the signed-message, if it is case, how Ironic get the message trust for Ceilometer, and send the valid message which can be accepted by Ceilometer Collector?


3. What is the Ceilometer sample data structure, and what is the min data item set for the IPMI message be emitted to Collector?
  name/type/unit/volume/timestamp/source - is this min data item set?
  
3. If the detailed data model should be defined for our IPMI data now?, what is our the first version scope, how many IPMI data type we should support? Here is a IPMI data sample list, I think we can support these as a min set.
  Temperature - System Temp/CPU Temp
  FAN Speed in rpm - FAN 1/2/3/4/A
  Volts - Vcore/3.3VCC/12V/VDIMM/5VCC/-12V/VBAT/VSB/AVCC


4. More specs - such as naming conversions, common constant reference definitions ...


Just a draft, correct me if I am wrong understanding and add the missing aspects, we can discuss these interface and data model clearly with several rounds:)




----------------------------------------------------------
Haomeng
Thanks:)





At 2013-11-21 16:08:00,"Ladislav Smola" <lsmola at redhat.com> wrote:

Responses inline.

On 11/20/2013 07:14 PM, Devananda van der Veen wrote:

Responses inline.


On Wed, Nov 20, 2013 at 2:19 AM, Ladislav Smola <lsmola at redhat.com> wrote:
Ok, I'll try to summarize what will be done in the near future for Undercloud monitoring.

1. There will be Central agent running on the same host(hosts once the central agent horizontal scaling is finished) as Ironic



Ironic is meant to be run with >1 conductor service. By i-2 milestone we should be able to do this, and running at least 2 conductors will be recommended. When will Ceilometer be able to run with multiple agents?

Here it is described and tracked: https://blueprints.launchpad.net/ceilometer/+spec/central-agent-improvement




On a side note, it is a bit confusing to call something a "central agent" if it is meant to be horizontally scaled. The ironic-conductor service has been designed to scale out in a similar way to nova-conductor; that is, there may be many of them in an AZ. I'm not sure that there is a need for Ceilometer's agent to scale in exactly a 1:1 relationship with ironic-conductor?

Yeah we have already talked about that. Maybe some renaming will be in place later. :-) I don't think it has to be 1:1 mapping. There was only requirement to have "Hardware agent" only on hosts with ironic-conductor, so it has access to management network, right?


 
2. It will have SNMP pollster, SNMP pollster will be able to get list of hosts and their IPs from Nova (last time I
    checked it was in Nova) so it can poll them for stats. Hosts to poll can be also defined statically in config file.



Assuming all the undercloud images have an SNMP daemon baked in, which they should, then this is fine. And yes, Nova can give you the IP addresses for instances provisioned via Ironic.
 

Yes.


3. It will have IPMI pollster, that will poll Ironic API, getting list of hosts and a fixed set of stats (basically everything
    that we can get :-))



No -- I thought we just agreed that Ironic will not expose an API for IPMI data. You can poll Nova to get a list of instances (that are on bare metal) and you can poll Ironic to get a list of nodes (either nodes that have an instance associated, or nodes that are unprovisioned) but this will only give you basic information about the node (such as the MAC addresses of its network ports, and whether it is on/off, etc).

Ok sorry I have misunderstood the:
"If there is a fixed set of information (eg, temp, fan speed, etc) that ceilometer will want,let's make a list of that and add a driver interface within Ironic to abstract the collection of that information from physical nodes. Then, each driver will be able to implement it as necessary for that vendor. Eg., an iLO driver may poll its nodes differently than a generic IPMI driver, but the resulting data exported to Ceilometer should have the same structure."

I thought I've read the data will be exposed, but it will be just internal Ironic abstraction, that will be polled by Ironic and send directly do Ceilometer collector. So same as the point 4., right? Yeah I guess this will be easier to implement.


 
4. Ironic will also emit messages (basically all events regarding the hardware) and send them directly to Ceilometer collector



Correct. I've updated the BP:


https://blueprints.launchpad.net/ironic/+spec/add-ceilometer-agent



Let me know if that looks like a good description.

Yeah, seems great. I would maybe remove the word 'Agent', seems Ironic will send it directly to Ceilometer collector, so Ironic acts as agent, right?



-Devananda






Does it seems to be correct? I think that is the basic we must have to have Undercloud monitored. We can then build on that.

Kind regards,
Ladislav





On 11/20/2013 09:22 AM, Julien Danjou wrote:
On Tue, Nov 19 2013, Devananda van der Veen wrote:

If there is a fixed set of information (eg, temp, fan speed, etc) that
ceilometer will want,
Sure, we want everything.

let's make a list of that and add a driver interface
within Ironic to abstract the collection of that information from physical
nodes. Then, each driver will be able to implement it as necessary for that
vendor. Eg., an iLO driver may poll its nodes differently than a generic
IPMI driver, but the resulting data exported to Ceilometer should have the
same structure.
I like the idea.

An SNMP agent doesn't fit within the scope of Ironic, as far as I see, so
this would need to be implemented by Ceilometer.
We're working on adding pollster for that indeed.

As far as where the SNMP agent would need to run, it should be on the
same host(s) as ironic-conductor so that it has access to the
management network (the physically-separate network for hardware
management, IPMI, etc). We should keep the number of applications with
direct access to that network to a minimum, however, so a thin agent
that collects and forwards the SNMP data to the central agent would be
preferable, in my opinion.
We can keep things simple by having the agent only doing that polling I
think. Building a new agent sounds like it will complicate deployment
again.









-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131125/9b38049c/attachment.html>