<br><br><div class="gmail_quote">On Fri, Nov 9, 2012 at 10:04 AM, Russell Bryant <span dir="ltr"><<a href="mailto:rbryant@redhat.com" target="_blank">rbryant@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Greetings,<br>
<br>
Dan Smith and I are getting pretty close to finishing the first stage of<br>
no-db-compute work for Grizzly. Specifically, that means these two things:<br>
<br>
1) Sending more data from the nova-api service to avoid db reads in<br>
nova-compute.<br>
<br>
2) Pulling db access out of nova virt drivers and having the only code<br>
in nova-compute that touches the db in nova/compute/manager.py.<br>
<br>
Removing the remaining db access is going to take some more drastic<br>
changes. This is what the majority of our discussion at the design<br>
summit was about. The idea of a db writer service (nova-sink) was<br>
discussed, but the majority opinion seemed to be that if we're going to<br>
make a big change to how nova-compute service, the "nova-sink" idea was<br>
not enough. I'd like to kick that discussion off again so that we can<br>
settle on a specific direction forward.<br>
<br>
<br>
Here's a proposal based on the suggestions from the design summit<br>
session. The intent is to generate some discussion around this so that<br>
we can keep writing code knowing that we're generally headed in a<br>
direction that others are happy with.<br>
<br>
How about we create a new service called "nova-compute-proxy". In<br>
short, I would envision it looking very much like "nova-sink" in the<br>
short term, but evolving into much more over time as we carefully rework<br>
how some operations are handled.<br>
<br>
This service would act as a proxy for nova-compute in a couple of ways.<br>
<br>
1) The nova-compute service would use it as a proxy to accomplish<br>
certain tasks, such as targeted operations that need database access.<br>
<br>
2) Over time, it would be used as a proxy for other services that wish<br>
to execute some sort of compute action. For example, instead of<br>
directly asking nova-compute to complete a long running operation, the<br>
nova-compute-proxy would take this operation and monitor its progress.<br>
<br>
<br>
Architecture notes:<br>
<br>
The nova-compute-proxy service must be horizontally scalable. You can<br>
run as many of them as needed to scale out. There is not a 1 to 1<br>
relationship with nova-compute services. While nova-compute services<br>
have ownership of specific instances, this is not true of<br>
nova-compute-proxy.<br>
<br>
<br>
Evolution:<br>
<br>
The most immediate short term goal is to remove database accesses from<br>
the nova-compute service. Certain database accesses will just be pushed<br>
into nova-compute-proxy and exposed as methods that can be executed<br>
using rpc.call(). One such example will be instance_update() since that<br>
is used in many places in nova-compute. I expect a lot of this to be<br>
done in the near term.<br>
<br>
Operations that are top candidates for being orchestrated by<br>
nova-compute-proxy are operations that cross multiple compute nodes,<br>
such as migrations and resizes. However, there may be benefit in doing<br>
this for other long running operations, such as starting an instance.<br>
We should look into doing as much of this as is practical in this<br>
development cycle, but I suspect much of this would carry over into H<br>
development.<br>
<br>
<br>
Some questions, complications, vagueness:<br>
<br>
a) There is still a bit of hand waving around the more significant<br>
functionality will work in nova-compute-proxy (something like resizes).<br>
It seems to make sense from a high level, but I haven't tried mapping<br>
out exact order of operations or anything. Does it seem to make sense,<br>
or do we need to dive deeper?<br>
<br>
b) How about periodic tasks? Right now nova-compute does a number of<br>
periodic tasks, mostly associated with cleanup for the local instances.<br>
If long term we want to simplify nova-compute, where do these go?<br>
Short term, for no-db-compute, we can leave them where they are and just<br>
use nova-compute-proxy for db access as needed. It feels like we need a<br>
good long term vision here though, and I'm not sure what it is.<br>
<br>
If nova-compute-proxy was directly tied to N instances of nova-compute,<br>
nova-compute-proxy could just do the periodic tasks. I was hoping we<br>
could not have to tie instances of the two services together because it<br>
seems to leave the architecture more flexible and less complicated.<br></blockquote><div><br></div><div>This sounds a lot like what nova baremetal is doing, where one service is managing many 'virt' machines.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Is the short-term approach for periodic tasks acceptable for now?<br>
<br>
What should the long term plan be? And does that have an impact on what<br>
we do right now?<br>
<br>
<br>
Thoughts?<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Russell Bryant<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</font></span></blockquote></div><br>