[openstack-dev] [ironic] using ironic as a replacement for existing datacenter baremetal provisioning

Kris G. Lindgren klindgren at godaddy.com
Mon Jun 6 20:44:26 UTC 2016


Hi ironic folks,
As I'm trying to explore how GoDaddy can use ironic I've created the following in an attempt to document some of my concerns, and I'm wondering if you folks could help myself identity ongoing work to solve these (or alternatives?)
List of concerns with ironic:

1.)Nova <-> ironic interactions are generally seem terrible?
  -How to accept raid config and partitioning(?) from end users? Seems to not a yet agreed upon method between nova/ironic.
   -How to run multiple conductors/nova-computes?   Right now as far as I can tell all of ironic front-end by a single nova-compute, which I will have to manage via a cluster technology between two or mode nodes.  Because of this and the way host-agregates work I am unable to expose fault domains for ironic instances (all of ironic can only be under a single AZ (the az that is assigned to the nova-compute node)). Unless I create multiple nova-compute servers and manage multiple independent ironic setups.  This makes on-boarding/query of hardware capacity painful.
  - Nova appears to be forcing a we are "compute" as long as "compute" is VMs, means that we will have a baremetal flavor explosion (ie the mismatch between baremetal and VM).
      - This is a feeling I got from the ironic-nova cross project meeting in Austin.  General exmaple goes back to raid config above. I can configure a single piece of hardware many different ways, but to fit into nova's world view I need to have many different flavors exposed to end-user.  In this way many flavors can map back to a single piece of hardware with just a lsightly different configuration applied. So how am I suppose to do a single server with 6 drives as either: Raid 1 + Raid 5, Raid 5, Raid 10, Raid 6, or JBOD.  Seems like I would need to pre-mark out servers that were going to be a specific raid level.  Which means that I need to start managing additional sub-pools of hardware to just deal with how the end users wants the raid configured, this is pretty much a non-starter for us.  I have not really heard of whats being done on this specific front.

2.) Inspector:
  - IPA service doesn't gather port/switching information
  - Inspection service doesn't process port/switching information, which means that it wont add it to ironic.  Which makes managing network swinging of the host a non-starter.  As I would inspect the host – then modify the ironic record to add the details about what port/switch the server is connected to from a different source.  At that point why wouldn't I just onboard everything through the API?
  - Doesn't grab hardware disk configurations, If the server has multiple raids (r1 + r5) only reports boot raid disk capacity.
  - Inspection is geared towards using a different network and dnsmasq infrastructure than what is in use for ironic/neutron.  Which also means that in order to not conflict with dhcp requests for servers in ironic I need to use different networks.  Which also means I now need to handle swinging server ports between different networks.

3.) IPA image:
  - Default build stuff is pinned to extremly old versions due to gate failure issues. So I can not work without a fork for onboard of servers due to the fact that IPMI modules aren't built for the kernel, so inspection can never match the node against ironic.  Seems like currently functionality here is MVP for gate to work and to deploy images.  But if you need to do firmware, bios-config, any other hardware specific features you are pretty much going to need to roll your own IPA image and IPA modules to do standard provisioning tasks.

4.) Conductor:
  - Serial-over-lan consoles require a unique port on the conductor server (I have seen purposes to try and fix this?), this is painful to manage with large numbers of servers.
  - SOL consoles aren't restarted when conductor is restarted (I think this might be fixed in newer versions of ironic?), again if end users aren't suppose to consume ironic api's directly - this is painful to handle.
  - Its very easy to get a node to fall off the staemachine rails (reboot a server while an image is being deployed to it), the only way I have seen to be able to fix this is to update the DB directly.
  - As far as I can tell shell-in-a- box, SOL consoles aren't support via nova – so how are end users suppose to consume the shell-in-box console?
  - I have BMC that need specific configuration (some require SOL on com2, others on com1) this makes it pretty much impossible without per box overrides against the conductor hardcoded templates.
  - Additionally it would be nice to default to having a provisioning kernel/image that was set as a single config option with per server overrides – rather than on each server.  If we ever change the IPA image – that means at scale we would need to update thousands of ironic nodes.

What is ironic doing to monitor the hardware for failures?  I assume the answer here is nothing and that we will need to make sure the images that we deploy are correctly configuring the tools to monitor disk/health/psu's/ram errors, ect. ect.

Overall the above concerns have me wondering if I am going crazy, or is ironic really not ready to take over datacenters baremetal provisioning (unless you have a very limited view of functionality that is needed in those datacenters).

Thoughts would be great,
___________________________________________________________________
Kris Lindgren
Senior Linux Systems Engineer
GoDaddy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160606/7f2326bf/attachment.html>


More information about the OpenStack-dev mailing list