Open Stack

Hi all!

As you know I actively promote ironic-discoverd project [1] as one of 
the means to do hardware inspection for Ironic (see e.g. spec [2]), so I 
decided it's worth to give some updates to the community from time to 
time. This email is purely informative, you may safely skip it, if 
you're not interested.

Background
==========

The discoverd project (I usually skip the "ironic-" part when talking 
about it) solves the problem of populating information about a node in 
Ironic database without help of any vendor-specific tool. This 
information usually includes Nova scheduling properties (CPU, RAM, disk 
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting data 
there and posting it back to discoverd HTTP API. Thus actually discoverd 
consists of 2 components: the service [1] and the ramdisk [3]. The 
service handles 2 major tasks:
* Processing data posted by the ramdisk, i.e. finding the node in Ironic 
database and updating node properties with new data.
* Managing iptables so that the default PXE environment for 
introspection does not interfere with Neutron

The project was born from a series of patches to Ironic itself after we 
discovered that this change is going to be too intrusive. Discoverd was 
actively tested as part of Instack [4] and it's RPM is a part of Juno 
RDO. After the Paris summit, we agreed on bringing it closer to the 
Ironic upstream, and now discoverd is hosted on StackForge and tracks 
bugs on Launchpad.

Future
======

The basic feature of discoverd: supply Ironic with properties required 
for scheduling, is pretty finished as of the latest stable series 0.2.

However, more features are planned for release 1.0.0 this January [5]. 
They go beyond the bare minimum of finding out CPU, RAM, disk size and 
NIC MAC's.

Plugability
~~~~~~~~~~~

An interesting feature of discoverd is support for plugins, which I 
prefer to call hooks. It's possible to hook into the introspection data 
processing chain in 2 places:
* Before any data processing. This opens opportunity to adopt discoverd 
to ramdisks that have different data format. The only requirement is 
that the ramdisk posts a JSON object.
* After a node is found in Ironic database and ports are created for 
MAC's, but before any actual data update. This gives an opportunity to 
alter, which properties discoverd is going to update.

Actually, even the default logic of update Node.properties is contained 
in a plugin - see SchedulerHook in ironic_discoverd/plugins/standard.py 
[6]. This plugability opens wide opportunities for integrating with 3rd 
party ramdisks and CMDB's (which as we know Ironic is not ;).

Enrolling
~~~~~~~~~

Some people have found it limiting that the introspection requires power 
credentials (IPMI user name and password) to be already set. The recent 
set of patches [7] introduces a possibility to request manual power on 
of the machine and update IPMI credentials via the ramdisk to the 
expected values. Note that support of this feature in the reference 
ramdisk [3] is not ready yet. Also note that this scenario is only 
possible when using discoverd directly via it's API, not via Ironic API 
like in [2].

Get Involved
============

Discoverd terribly lacks reviews. Out team is very small and 
self-approving is not a rare case. I'm even not against fast-tracking 
any existing Ironic core to a discoverd core after a couple of 
meaningful reviews :)

And of course patches are welcome, especially plugins for integration 
with existing systems doing similar things and CMDB's. Patches are 
accepted via usual Gerrit workflow. Ideas are accepted as Launchpad 
blueprints (we do not follow the Gerrit spec process right now).

Finally, please comment on the Ironic spec [2], I'd like to know what 
you think.

References
==========

[1] https://pypi.python.org/pypi/ironic-discoverd
[2] https://review.openstack.org/#/c/135605/
[3] 
https://github.com/openstack/diskimage-builder/tree/master/elements/ironic-discoverd-ramdisk
[4] https://github.com/agroup/instack-undercloud/
[5] https://bugs.launchpad.net/ironic-discoverd/+milestone/1.0.0
[6] 
https://github.com/stackforge/ironic-discoverd/blob/master/ironic_discoverd/plugins/standard.py
[7] 
https://blueprints.launchpad.net/ironic-discoverd/+spec/setup-ipmi-credentials