Consider Hadoop as one instance of what we are willing to support. The instantiation of a Hadoop cluster involve some placement constraints, like not placing two Task Trackers on the same physical host. Sahara accomplish that by defining anti-affinity groups, so I was wondering if there would be a more direct way to place VMs that wouldn`t even require hinting simplistic policies to the scheduler. Can`t we simply say: "create VM X on host Y"?

On Mon, Nov 24, 2014 at 6:53 PM, Narayan Desai <narayan.desai@gmail.com> wrote:

This doesn't seem like a good match for openstack's model. If you want data locality, you want persistence of data and a resource allocation that has a relatively long lifetime, but your tasks will be short. Since openstack largely deals with coarse grained allocations, you probably aren't in for a good time if you want to build a fine-grained adaptive system on top of the openstack scheduler.

Have you looked at mesos? I think it might be better infrastructure for your use case.

-nld

On Mon, Nov 24, 2014 at 7:40 PM, Phab Lucky <phab.lucky@gmail.com> wrote:

Hi, all.

I'm new to OpenStack, so I apologize in advance if my question lacks relevance or sense.

We are trying to build a cluster orchestrator for Big Data processing and one of the key requirements is the ability to place each computing task in each physical server. We have been trying to leverage this development with OpenStack, but, as far as I can see by the on-line documentation, the Nova interface does not allow users to specify the physical location (host) where VMs (servers) are created. Instead, this task is performed by the Scheduler, which has its own algorithms and policies.

Even though we can try to customize the Scheduler, I was wondering if I could simply bypass it and tell Nova to create a VM on host X. I know this sounds like bad design, as we are violating the location transparency that OpenStack attempts to deliver, but this seems to me the only way to bring the computation to where the data is.

Any thoughts?

Thanks a lot.

Phab

_______________________________________________
OpenStack-HPC mailing list
OpenStack-HPC@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc