Open Stack

Excerpts from Steven Hardy's message of 2014-09-15 10:10:05 -0700:
> On Mon, Sep 15, 2014 at 09:50:24AM -0700, Clint Byrum wrote:
> > Excerpts from Steven Hardy's message of 2014-09-15 04:44:24 -0700:
> > > All,
> > > 
> > > Starting this thread as a follow-up to a strongly negative reaction by the
> > > Ironic PTL to my patches[1] adding initial Heat->Ironic integration, and
> > > subsequent very detailed justification and discussion of why they may be
> > > useful in this spec[2].
> > > 
> > > Back in Atlanta, I had some discussions with folks interesting in making
> > > "ready state"[3] preparation of bare-metal resources possible when
> > > deploying bare-metal nodes via TripleO/Heat/Ironic.
> > > 
> > > The initial assumption is that there is some discovery step (either
> > > automatic or static generation of a manifest of nodes), that can be input
> > > to either Ironic or Heat.
> > > 
> > > Following discovery, but before an undercloud deploying OpenStack onto the
> > > nodes, there are a few steps which may be desired, to get the hardware into
> > > a state where it's ready and fully optimized for the subsequent deployment:
> > > 
> > > - Updating and aligning firmware to meet requirements of qualification or
> > >   site policy
> > > - Optimization of BIOS configuration to match workloads the node is
> > >   expected to run
> > > - Management of machine-local storage, e.g configuring local RAID for
> > >   optimal resilience or performance.
> > > 
> > > Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
> > > of these steps possible, but there's no easy way to either encapsulate the
> > > (currently mostly vendor specific) data associated with each step, or to
> > > coordinate sequencing of the steps.
> > > 
> > 
> > First, Ironic is hidden under Nova as far as TripleO is concerned. So
> > mucking with the servers underneath Nova during deployment is a difficult
> > proposition. Would I look up the Ironic node ID of the nova server,
> > and then optimize it for the workload after the workload arrived? Why
> > wouldn't I just do that optimization before the deployment?
> 
> That's exactly what I'm proposing - a series of preparatory steps performed
> before the node is visible to nova, before the deployment.
> 

Ok good, so I didn't misunderstand. I'm having trouble seeing where Heat
is a good fit there.

> The whole point is that Ironic is hidden under nova, and provides no way to
> perform these pre-deploy steps via interaction with nova.
> 
> > 
> > > What is required is some tool to take a text definition of the required
> > > configuration, turn it into a correctly sequenced series of API calls to
> > > Ironic, expose any data associated with those API calls, and declare
> > > success or failure on completion.  This is what Heat does.
> > > 
> > 
> > I'd rather see Ironic define or adopt a narrow scope document format
> > that it can consume for bulk loading. Heat is extremely generic, and thus
> > carries a ton of complexity for what is probably doable with a CSV file.
> 
> Perhaps you can read the spec - it's not really about the bulk-load part,
> it's about orchestrating the steps to prepare the node, after it's
> registered with Ironic, but before it's ready to have the stuff deployed to
> it.
> 

Sounds like workflow to me. :-P

> What tool do you think will "just do that optimization before the
> deployment"? (snark not intended, I genuinely want to know, is it scripts
> in TripleO, some sysadmin pre-deploy steps, magic in Ironic?)
>

If it can all be done by calls to the ironic client with the node ID and
parameters from the user, I'd suggest that this is a simple workflow
and can be done in the step prior to 'heat stack-create'. I don't see
any reason to keep a bunch of records around in Heat to describe what
happened, identically, for Ironic nodes. It is an ephemeral step in the
evolution of the system, not something we need to edit on a regular basis.

My new bar for whether something is a good fit for Heat is what happens
to my workload when I update it. If I go into my Ironic pre-registration
stack and change things around, the likely case is that my box reboots
to re-apply BIOS updates with the new paramters. And there is a missing
dependency expression when using the orchestration tool to do the
workflow job. It may actually be necessary to always do these things to
the hardware in a certain sequence. But editting the Heat template and
updating has no way to express that.

To contrast this with developing it in a workflow control language
(like bash), it is imperative so I am consciously deciding to re-apply
those things by running it. If I only want to do one step, I just do
the one step.

Basically, the imperative model is rigid, sharp, and pointy, but the
declarative model is soft and maleable, and full of unexpected sharp
pointy things. I think users are more comfortable with knowing where
the sharp and pointy things are, than stumbling on them.

> > > So the idea is to create some basic (contrib, disabled by default) Ironic
> > > heat resources, then explore the idea of orchestrating ready-state
> > > configuration via Heat.
> > > 
> > > Given that Devananda and I have been banging heads over this for some time
> > > now, I'd like to get broader feedback of the idea, my interpretation of
> > > "ready state" applied to the tripleo undercloud, and any alternative
> > > implementation ideas.
> > > 
> > 
> > I think there may be value in being able to tie Ironic calls to other
> > OpenStack API calls. I'm dubious that this is an important idea, but
> > I think if somebody wants to step forward with their use case for it,
> > then the resources might make sense. However, I realy don't see the
> > _enrollment_ phase as capturing any value that isn't entirely offset by
> > added complexity.
> 
> I've stepped forward with a use-case for it, in the spec, and it's not
> about the enrollment phase (although Heat could do that, if for example it
> was used to manage the series of steps required for autodiscovery).
> 

Yeah I think I got confused by the talk of it in other branches of the
thread. I think I understand now that this is a post-enrollment
workflow.

> If there's a better way to do this, I'm very happy to hear about it, but I
> am getting a little tired of folks telling me the idea is dubious without
> offering any alternative solution to the requirement :(

Any workflow tool will do this just fine. I do think Heat is an o-k choice
for this, but I'd rather see TripleO embrace more workflow control via
API, than bend Heat into that role.