Hi Jonathon,
We are in a similar position at UAB and investigating a solution with openstack managed hw-level provisioning as well. I'm interested in being able to instantiate a cluster via OpenStack on select hardware fabrics using Ironic.
We currently have a traditional ROCKS HPC fabric augmented by Kickstart and Puppet for additional services (NFS servers, monitoring, documentation, project tools etc) spread out over hardware and VMs. We began adding VM services into the cluster to support ancillary tools and newer use cases like web-front ends to HPC job submission for select science domains.
We added an OpenStack fabric in 2013 to simplify such services and make them available to end users to allow greater flexibility and autonomy. it has been invaluable as a way to instantiate new services and other test configurations quickly and easily. This page gives a bit of background and and overview.
https://dev.uabgrid.uab.edu/wiki/OpenStackPlusCeph
My goal is to have a generally available openstack fabric that can serve our research community with Amazon-like functionality, allowing them to build their own scale outs if desired and potentially even help us manage the "HPC engine" (replacing the ROCKS mode with more cloud-oriented provisioning). There's another thread that is looking to Puppet+Foreman to handle the hardware provisioning -- building on our puppet effort. We also have a Chef+Crowbar fabric that provides similar services for our existing OpenStack deploy.
Right now we are planning our migration from Essex to Juno++. This is presenting it's own challenges but gives us a chance to build from a clean foundation. I haven't played with Ironic yet but it is definitely a tool I am interested in.
I'd like to be able to support VM containers for HPC jobs but can't determine yet if these should run inside the OpenStack provisioned cluster as traditional HPC-like jobs or allocate resources to them from raw hardware pool or as VMs (depending on requirements). It's probably a matter of perspective.
~jpr
On 01/13/2015 01:13 PM, Jonathon A Anderson wrote:
Hi, everybody!
We at CU-Boulder Research Computing are looking at using OpenStack Ironic for our HPC cluster(s). I’ve got a sysadmin background, so my initial goal is to simply use OpenStack to deploy our traditional queueing system (Slurm), and basically use OpenStack (w/Ironic) as a replacement for Cobbler, xcat, perceus, hand-rolled PXE, or whatever else sysadmin-focused PXE-based node provisioning solution one might use. That said, once we have OpenStack (and have some competency with it), we’d presumably be well positioned to offer actual cloud-y virtualization to our users, or even offer baremetal instances for direct use for some of our (presumably most trusted) users.
But how about the rest of you? Are you using OpenStack in your HPC workloads today? Are you doing “traditional” virtualization? Are you using the nova-compute baremetal driver? Are you using Ironic? Something even more exotic?
And, most importantly, how has it worked for you?
I’ve finally got some nodes to start experimenting with, so I hope to have some real hands-on experience with Ironic soon. (So far, I’ve only been able to do traditional virtualization on some workstations.)
~jonathon
OpenStack-HPC mailing list OpenStack-HPC@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc