<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, May 14, 2013 at 1:49 PM, Russell Bryant <span dir="ltr"><<a href="mailto:rbryant@redhat.com" target="_blank">rbryant@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im">On 05/13/2013 12:58 PM, Dan Wendlandt wrote:<br>
><br>
><br>
><br>
> On Fri, May 10, 2013 at 11:36 AM, Russell Bryant <<a href="mailto:rbryant@redhat.com">rbryant@redhat.com</a><br>
</div><div class="im">> <mailto:<a href="mailto:rbryant@redhat.com">rbryant@redhat.com</a>>> wrote:<br><br>
><br>
><br>
> I may be at fault here for introducing the work "proxy", but I think it<br>
> may be confusing things a bit, as many cases where you would want to use<br>
> something like vCenter are not really a proxy.<br>
><br>
> They way I see it, there are two extremes:<br>
> 1) The current virt-driver approach, with one nova-compute per per<br>
> "host", where "host" is a single unit of capacity in terms of<br>
> scheduling, etc. In KVM-world a "host" is a hypervisor node. In<br>
> current vCenter driver, this is a cluster, with vCenter exposing one<br>
> large capacity and spreading workloads evenly. This approach leverages<br>
> all scheduling logic available within nova.scheduler, uses nova DB<br>
> model, etc.<br>
<br>
</div>I would add that we assume that Nova is in complete control of the<br>
compute resources in this case, meaning that there is not another system<br>
making changes to instances. That's where we start to run into problems<br>
with putting the cluster-based drivers at this level.<br></blockquote><div><br></div><div style>I think we agree here, or at least that anything done to the compute resource should be transparent to Nova. For example, if libvirt auto-starts the VM on a host reboot, that is OK, as it is transport to Nova. You example below about the volume support would be an example of a change that is not transparent to Nova/Cinder. Is that inline with your thinking? </div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br>
> 2) A true "API proxy" approach, possibly implemented using cells. All<br>
> scheduling/placement, data modeling, etc. logic would be implemented by<br>
> a back-end system such as vCenter and one cannot leverage existing nova<br>
> scheduler logic or database models. I think this would mean that the<br>
> nova-api, nova-scheduler, and nova-compute code used with the<br>
> virt-driver model would not be used, and the new cell driver would have<br>
> to create its own versions of this.<br>
<br>
</div>I would actually break this up into two cases.<br>
<br>
2.a) A true API proxy. You have an existing virt management solution<br>
(vCenter, oVirt, whatever), and you want to interact with it using the<br>
OpenStack APIs. For this, I would propose not using Nova (or any<br>
OpenStack component) at all. Instead, I would implement the API in the<br>
project/product itself, or use something built to be an API proxy, like<br>
deltacloud. </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
2.b) A cell-based nova deployment. A cell may be a compute cell (what<br>
exists today) where Nova is managing all of the compute resources.<br>
(Here is where the proposal comes in) A cell could also be a different,<br>
existing virt management solution. In that case, the other system is<br>
responsible for everything that a compute cell does today, but does it<br>
its own way and is responsible for reporting state up to the API cell.<br>
Systems would of course be welcome to reuse Nova components if<br>
applicable, such as nova-scheduler.<br></blockquote><div><br></div><div style>Yeah, i agree. </div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="im"><br>
> is actually something<br>
> in between these two extremes and in fact closer to the virt-driver<br>
> model. I suspect the proposal sees the following benefits to a model<br>
> that is closer to the existing virt-driver (note: I am not working<br>
> closely with the author, so I am guessing here based on my own views):<br>
> - the nova scheduler logic is actually very beneficial even when you<br>
> are using something like vCenter. It lets you do a KVM + vmware<br>
> deployment, where workloads are directed to vmware vs. KVM by the<br>
> scheduler based on disk type, host aggregates, etc. It also lets you<br>
> expose different vCenter clusters with different properties via host<br>
> aggregates (e.g., an HA cluster and a non-HA cluster). According to the<br>
> docs I've read on cells (may be out of date), it seems like the current<br>
> cell scheduler is very simple (i.e., random), so doing this with cells<br>
> would require adding similar intelligence at the cell scheduler layer.<br>
> Additionally, I'm aware of people who would like to use nova's pluggable<br>
> scheduling to even do fine-grain per-hypervisor scheduling on a<br>
> "cluster" platform like vCenter (which for a cluster with DRS enabled<br>
> would make sense).<br>
<br>
</div>You could re-use the host scheduler in your cell if you really wanted to.<br>
<br>
The cell scheduler will have filter/weight support just like the host<br>
scheduler very soon, so we should be able to have very intelligent cell<br>
scheduling, just like host scheduling.<br>
<br>
<a href="https://review.openstack.org/#/c/16221/" target="_blank">https://review.openstack.org/#/c/16221/</a></blockquote><div><br></div><div style>Ok, that makes sense. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<div class="im"><br></div>
<br>
Like I mentioned before, my intent is to look beyond this blueprint. We<br>
need to identify where we want to go so that the line can be drawn<br>
appropriately in the virt driver layer for future proposals.<br></blockquote><div><br></div><div style>Yup, I think we agree that this is the key question. Some back-ends will require changes to the virt-layer that are natural extensions (e.g., scheduling across host, node pairs as you mentioned) and some back-ends will want to do something that is so fundamentally different that it doesn't make sense to unaturally contort the virt-driver API to match it, at which point we need a different plan (e.g., cells). In my last email I was just trying to say that splitting host, node for scheduling struck me as a reasonable extension of the existing virt-driver api, but it sounds like my nova knowledge is out of date and that has happened already :) </div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
There are other blueprints that perhaps do a better job at demonstrating<br>
the problems with putting these systems at the existing virt layer. As<br>
an example, check out the vCenter blueprint related to volume support [2].<br>
<br>
To make things work, this blueprint proposes that Nova gets volume<br>
connection info *for every possible host* that the volume may end up<br>
getting connected to. Presumably this is because the system wants to<br>
move things around on its own. This feels very, very wrong to me.<br>
<br>
If another system wants to manage the VMs, that's fine, but I'd rather<br>
Nova not *also* think it's in charge, which is what we have right now.<br></blockquote><div><br></div><div style> I need to learn more about Nova/Cinder integration in general, but at a high-level I agree that such an approach seems to be closer to the "unnatural contortion" side of things. </div>
<div style><br></div><div style>Dan</div><div style> </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
[1]<br>
<a href="https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service" target="_blank">https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service</a><br>
[2]<br>
<a href="https://blueprints.launchpad.net/nova/+spec/fc-support-for-vcenter-driver" target="_blank">https://blueprints.launchpad.net/nova/+spec/fc-support-for-vcenter-driver</a><br>
<div class=""><div class="h5"><br>
--<br>
Russell Bryant<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Dan Wendlandt <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br><div>twitter: danwendlandt<br>
~~~~~~~~~~~~~~~~~~~~~~~~~~~<br></div></div>
</div></div>