This doesn't seem like a good match for openstack's model. If you want data locality, you want persistence of data and a resource allocation that has a relatively long lifetime, but your tasks will be short. Since openstack largely deals with coarse grained allocations, you probably aren't in for a good time if you want to build a fine-grained adaptive system on top of the openstack scheduler.

Have you looked at mesos? I think it might be better infrastructure for your use case.

-nld

On Mon, Nov 24, 2014 at 7:40 PM, Phab Lucky <phab.lucky@gmail.com> wrote:

Hi, all.

I'm new to OpenStack, so I apologize in advance if my question lacks relevance or sense.

We are trying to build a cluster orchestrator for Big Data processing and one of the key requirements is the ability to place each computing task in each physical server. We have been trying to leverage this development with OpenStack, but, as far as I can see by the on-line documentation, the Nova interface does not allow users to specify the physical location (host) where VMs (servers) are created. Instead, this task is performed by the Scheduler, which has its own algorithms and policies.

Even though we can try to customize the Scheduler, I was wondering if I could simply bypass it and tell Nova to create a VM on host X. I know this sounds like bad design, as we are violating the location transparency that OpenStack attempts to deliver, but this seems to me the only way to bring the computation to where the data is.

Any thoughts?

Thanks a lot.

Phab

_______________________________________________
OpenStack-HPC mailing list
OpenStack-HPC@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc