Two good questions.... my take:
Do you think this is something that would be useful to your users?
Probably. Though we have a pretty mature federated HPC environment with cloud-bursting capabilities which meets the needs of most of our internal users. We do run into situations where we need to stand up a "standalone" environment for a specific project or a short term research workflow. Being able to use Starcluster to deploy on an internal Openstack based private cloud "quick and dirty" would be handy.
To enable full functionality in the Horizon dashboard the dashboard app needs access to a private key to access the running (virtual) cluster nodes as root.....
This one is more of a challenge in my mind. Is full Horizon dashboard support worthwhile? Heck yes! Especially in an environment like ours where the users tend to be chemists and biologists, not computer scientists, having a web frontend for provisioning is extremely useful. And, like Jon, our horizon node is pretty well protected and I'm not too concerned caching keys. I'd be less enthused in a public cloud, but within the constraints of a properly implemented and secured private OpenStack implementation I don't see it as a show stopper at all. -------- Eric D Christensen R&D IS - Infrastructure and High Performance Computing Sanofi Tucson Innovation Center “We are what we repeatedly do. Excellence, therefore, is not an act, but a habit.” – Aristotle -----Original Message----- From: Jonathan Proulx [mailto:jon@jonproulx.com] Sent: Friday, April 18, 2014 1:39 PM To: openstack-hpc@lists.openstack.org; Justin Riley; Steve Jahl Subject: [openstack-hpc] Questions about porting StarCluster to OpenStack Hi All, For those who don't know me I deployed and run the OpenStack cloud at MIT CSAIL (http://www.csail.mit.edu) I'm currently working with Justin Riley (address in the "To" header of this email) who's the primary developer for StarCluster [http://star.mit.edu/cluster/index.html] on porting StarCluster to OpenStack. We have some operational and use case questions before we get too far down the implementation rathole^H^H^H^H^H^H^H path. I've sent this to openstack-hpc and some select bcc's to get a sense of who might be interest and what opinions you have about how the port should be implemented. What is it? If your not familiar, StarCluster is is an open source cluster-computing toolkit for Amazon’s Elastic Compute Cloud (EC2) released under the LGPL license. It has been designed to automate and simplify the process of building, configuring, and managing clusters of virtual machines on Amazon’s EC2 cloud. StarCluster allows anyone to easily create a cluster computing environment in the cloud suited for distributed and parallel computing applications and systems. It's main target audience is domain scientists who want to setup a SGE, Condor, Hadoop, and a few other sorts of cluster in "the cloud". Its current implementation is basically a config file driven CLI that uses locally stored ssh keys for managing running the cluster. It has a fair sized user community and people seem to like it, which is why one of my users introduced Justin and I so I could provide a place for Justin to work and my user could get StarCluster on our private cloud. Were is it at? The CLI "basically works" on OpenStack now, though so far no end-users have touched it so not quite ready for public beta. If you're really interested in early code I'm sure Justin will be happy to share if you ask nicely. He has also been working on a Horizon dashboard which would be an additional feature for the OpenStack version (there is no EC2 GUI). The BIG QUESTIONS? Do you think this is something that would be useful to your users? To enable full functionality in the Horizon dashboard the dashboard app needs access to a private key to access the running (virtual) cluster nodes as root. Waving hands over implementation details, and assuming you're interested in this functionality of you're this far into the email. Is storing key material a show stopper in your environment? In other words would you rather just fall back to the CLI with it's local ~/.ssh/id_rsa (or equivalent) for privileged operations? My Opinion... I don't like centrally storing crypto keys, but my horizon runs on my controller node so it is already a fairly privileged zone, and I don't necessarily trust my users to store their key material even as well as this could be done. So while a bit cautious about the details I think it is an acceptable risk in my (admittedly permissive) environment. Thoughts? -Jon _______________________________________________ OpenStack-HPC mailing list OpenStack-HPC@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc