Fwd: Scheduler Bypass
Hi, all. I'm new to OpenStack, so I apologize in advance if my question lacks relevance or sense. We are trying to build a cluster orchestrator for Big Data processing and one of the key requirements is the ability to place each computing task in each physical server. We have been trying to leverage this development with OpenStack, but, as far as I can see by the on-line documentation, the Nova interface does not allow users to specify the physical location (host) where VMs (servers) are created. Instead, this task is performed by the Scheduler, which has its own algorithms and policies. Even though we can try to customize the Scheduler, I was wondering if I could simply bypass it and tell Nova to create a VM on host X. I know this sounds like bad design, as we are violating the location transparency that OpenStack attempts to deliver, but this seems to me the only way to bring the computation to where the data is. Any thoughts? Thanks a lot. Phab
This doesn't seem like a good match for openstack's model. If you want data locality, you want persistence of data and a resource allocation that has a relatively long lifetime, but your tasks will be short. Since openstack largely deals with coarse grained allocations, you probably aren't in for a good time if you want to build a fine-grained adaptive system on top of the openstack scheduler. Have you looked at mesos? I think it might be better infrastructure for your use case. -nld On Mon, Nov 24, 2014 at 7:40 PM, Phab Lucky <phab.lucky@gmail.com> wrote:
Hi, all.
I'm new to OpenStack, so I apologize in advance if my question lacks relevance or sense.
We are trying to build a cluster orchestrator for Big Data processing and one of the key requirements is the ability to place each computing task in each physical server. We have been trying to leverage this development with OpenStack, but, as far as I can see by the on-line documentation, the Nova interface does not allow users to specify the physical location (host) where VMs (servers) are created. Instead, this task is performed by the Scheduler, which has its own algorithms and policies.
Even though we can try to customize the Scheduler, I was wondering if I could simply bypass it and tell Nova to create a VM on host X. I know this sounds like bad design, as we are violating the location transparency that OpenStack attempts to deliver, but this seems to me the only way to bring the computation to where the data is.
Any thoughts?
Thanks a lot.
Phab
_______________________________________________ OpenStack-HPC mailing list OpenStack-HPC@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc
Hi, Narayan. Thanks for your reply. I think my requirements are not as strict as for a fine-grained platform. Consider Hadoop as one instance of what we are willing to support. The instantiation of a Hadoop cluster involve some placement constraints, like not placing two Task Trackers on the same physical host. Sahara accomplish that by defining anti-affinity groups, so I was wondering if there would be a more direct way to place VMs that wouldn`t even require hinting simplistic policies to the scheduler. Can`t we simply say: "create VM X on host Y"? Thanks again. Phab On Mon, Nov 24, 2014 at 6:53 PM, Narayan Desai <narayan.desai@gmail.com> wrote:
This doesn't seem like a good match for openstack's model. If you want data locality, you want persistence of data and a resource allocation that has a relatively long lifetime, but your tasks will be short. Since openstack largely deals with coarse grained allocations, you probably aren't in for a good time if you want to build a fine-grained adaptive system on top of the openstack scheduler.
Have you looked at mesos? I think it might be better infrastructure for your use case. -nld
On Mon, Nov 24, 2014 at 7:40 PM, Phab Lucky <phab.lucky@gmail.com> wrote:
Hi, all.
I'm new to OpenStack, so I apologize in advance if my question lacks relevance or sense.
We are trying to build a cluster orchestrator for Big Data processing and one of the key requirements is the ability to place each computing task in each physical server. We have been trying to leverage this development with OpenStack, but, as far as I can see by the on-line documentation, the Nova interface does not allow users to specify the physical location (host) where VMs (servers) are created. Instead, this task is performed by the Scheduler, which has its own algorithms and policies.
Even though we can try to customize the Scheduler, I was wondering if I could simply bypass it and tell Nova to create a VM on host X. I know this sounds like bad design, as we are violating the location transparency that OpenStack attempts to deliver, but this seems to me the only way to bring the computation to where the data is.
Any thoughts?
Thanks a lot.
Phab
_______________________________________________ OpenStack-HPC mailing list OpenStack-HPC@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc
I don't know if this is what you want, but you can check the zone and hostname of your compute nodes with the command
nova service-list
and boot VMs on specific computes nodes with the following command:
nova boot --image <id_of_image> --flavor m1.large --availability-zone nova_zone:compute_hostname --key_name YourKey Name_Of_New_VM
However, you will most likely need administrative privileges for something like this. Vangelis On 11/25/2014 06:36 PM, Phab Lucky wrote:
Hi, Narayan.
Thanks for your reply. I think my requirements are not as strict as for a fine-grained platform.
Consider Hadoop as one instance of what we are willing to support. The instantiation of a Hadoop cluster involve some placement constraints, like not placing two Task Trackers on the same physical host. Sahara accomplish that by defining anti-affinity groups, so I was wondering if there would be a more direct way to place VMs that wouldn`t even require hinting simplistic policies to the scheduler. Can`t we simply say: "create VM X on host Y"?
Thanks again.
Phab On Mon, Nov 24, 2014 at 6:53 PM, Narayan Desai <narayan.desai@gmail.com <mailto:narayan.desai@gmail.com>> wrote:
This doesn't seem like a good match for openstack's model. If you want data locality, you want persistence of data and a resource allocation that has a relatively long lifetime, but your tasks will be short. Since openstack largely deals with coarse grained allocations, you probably aren't in for a good time if you want to build a fine-grained adaptive system on top of the openstack scheduler.
Have you looked at mesos? I think it might be better infrastructure for your use case. -nld
On Mon, Nov 24, 2014 at 7:40 PM, Phab Lucky <phab.lucky@gmail.com <mailto:phab.lucky@gmail.com>> wrote:
Hi, all.
I'm new to OpenStack, so I apologize in advance if my question lacks relevance or sense.
We are trying to build a cluster orchestrator for Big Data processing and one of the key requirements is the ability to place each computing task in each physical server. We have been trying to leverage this development with OpenStack, but, as far as I can see by the on-line documentation, the Nova interface does not allow users to specify the physical location (host) where VMs (servers) are created. Instead, this task is performed by the Scheduler, which has its own algorithms and policies.
Even though we can try to customize the Scheduler, I was wondering if I could simply bypass it and tell Nova to create a VM on host X. I know this sounds like bad design, as we are violating the location transparency that OpenStack attempts to deliver, but this seems to me the only way to bring the computation to where the data is.
Any thoughts?
Thanks a lot.
Phab
_______________________________________________ OpenStack-HPC mailing list OpenStack-HPC@lists.openstack.org <mailto:OpenStack-HPC@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc
_______________________________________________ OpenStack-HPC mailing list OpenStack-HPC@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-hpc
participants (3)
-
Narayan Desai
-
Phab Lucky
-
Vangelis Tasoulas