Open Stack

On 07/11/2014 07:14 AM, John Garbutt wrote:
> On 10 July 2014 16:59, Sylvain Bauza <sbauza at redhat.com> wrote:
>> Le 10/07/2014 15:47, Russell Bryant a écrit :
>>> On 07/10/2014 05:06 AM, Sylvain Bauza wrote:
>>>> Hi all,
>>>>
>>>> === tl;dr: Now that we agree on waiting for the split prereqs
>>>> to be done, we debate on if ResourceTracker should be part of
>>>> the scheduler code and consequently Scheduler should expose
>>>> ResourceTracker APIs so that Nova wouldn't own compute nodes
>>>> resources. I'm proposing to first come with RT as Nova
>>>> resource in Juno and move ResourceTracker in Scheduler for K,
>>>> so we at least merge some patches by Juno. ===
>>>>
>>>> Some debates occured recently about the scheduler split, so I
>>>> think it's important to loop back with you all to see where we
>>>> are and what are the discussions. Again, feel free to express
>>>> your opinions, they are welcome.
>>> Where did this resource tracker discussion come up?  Do you have
>>> any references that I can read to catch up on it?  I would like
>>> to see more detail on the proposal for what should stay in Nova
>>> vs. be moved.  What is the interface between Nova and the
>>> scheduler here?
>>
>> Oh, missed the most important question you asked. So, about the
>> interface in between scheduler and Nova, the original agreed
>> proposal is in the spec https://review.openstack.org/82133
>> (approved) where the Scheduler exposes : - select_destinations() :
>> for querying the scheduler to provide candidates -
>> update_resource_stats() : for updating the scheduler internal state
>> (ie. HostState)
>>
>> Here, update_resource_stats() is called by the ResourceTracker,
>> see the implementations (in review)
>> https://review.openstack.org/82778 and
>> https://review.openstack.org/104556.
>>
>> The alternative that has just been raised this week is to provide
>> a new interface where ComputeNode claims for resources and frees
>> these resources, so that all the resources are fully owned by the
>> Scheduler. An initial PoC has been raised here
>> https://review.openstack.org/103598 but I tried to see what would
>> be a ResourceTracker proxified by a Scheduler client here :
>> https://review.openstack.org/105747. As the spec hasn't been
>> written, the names of the interfaces are not properly defined but
>> I made a proposal as : - select_destinations() : same as above -
>> usage_claim() : claim a resource amount - usage_update() : update
>> a resource amount - usage_drop(): frees the resource amount
>>
>> Again, this is a dummy proposal, a spec has to written if we
>> consider moving the RT.
>
> While I am not against moving the resource tracker, I feel we could
> move this to Gantt after the core scheduling has been moved.

Big -1 from me on this, John.

Frankly, I see no urgency whatsoever -- and actually very little benefit
-- to moving the scheduler out of Nova. The Gantt project I think is
getting ahead of itself by focusing on a split instead of focusing on
cleaning up the interfaces between nova-conductor, nova-scheduler, and
nova-compute.

I see little to no long-term benefit in splitting the scheduler --
especially with its current design -- out from Nova. It's not like
Neutron or Cinder, where the split-out service is providing management
of a particular kind of resource (network, block storage). The Gantt
project isn't providing any resource itself. Instead, it would be acting
as a proxy for the placement other services' resources, which, IMO, in
and of itself, is not a reason to go through the trouble of splitting
the scheduler out of Nova.

> I was imagining the extensible resource tracker to become (sort of)
> equivalent to cinder volume drivers.

The problem with the extensible resource tracker design is that it,
again, just shoves a bunch of stuff into a JSON field and both the
existing resource tracker code (in nova-compute) as well as the
scheduler code (in nova-scheduler) then need to use and abuse this BLOB
of random data.

I tried to make my point on the extensible resource tracker blueprint
about my objections to the design.

My first, and main, objection is that there was never demonstrated ANY
clear use case for the extensibility of resources that was not already
covered by the existing resource tracker and scheduler. The only "use
case" was a vague "we have out of tree custom plugins that depend on
divergent behaviour and therefore we need a plugin point to change the 
way the scheduler thinks of a particular resource." And that is not a 
use case. It's simply a request to break compatibility between clouds 
with out-of-tree source code.

My second objection was that the proposed implementation added yet more
completely unnecessary complication to the scheduler, with the
introduction of "scheduler consumers", one for each "extensible"
resource added as plugins to the resource tracker. The existing
scheduler is already displaying a silly amount of needless complexity,
and current (and approved) blueprints like this one merely add to the
endless array of configuration options and ostensibly flexible behaviour.

The problem with ALL of these approved and in-review specs, IMO, is that
none of them have any clear use cases that show the newly-added
configuration options (things like yet more "weigher multipliers") or
plugins (scheduler consumers, resource plugins) actually fulfill a real
user's needs. I've yet to meet an operator that actually uses scheduler
configuration options outside a very small variance from the default
configuration. No admin or operator is sitting there tinkering with the
"weigher multipliers" and the like. All of these blueprints seem to
me to be either an academic exercise or an effort to allow out-of-tree,
custom, divergent code to be run against the upstream Nova codebase.

If I am wrong, please somebody explain a clear use case for these things
that does not involve "we want to run our divergent code".

> Also the persistent resource claims will give us another plugin point
> for gantt. That might not be enough, but I think it easier to see
> once the other elements have moved.

Is there some code you can provide me a link to for these persistent
resource claims? Not sure I've seen that code yet... or a blueprint?

As for the idea that things will get *easier* once scheduler code is
broken out of Nova, I go back to my original statement that I don't
really see the benefit of the split at this point, and I would just
bring up the fact that Neutron/nova-network is a shining example of how
things can easily backfire when splitting of code is done too early
before interfaces are cleaned up and responsibilities between internal
components are not clearly agreed upon.

I don't think we should be afraid to change course on the Gantt effort
just because we've talked about (and agreed in the past) to a split for
a long time now.

Best,
-jay

> But the key point thing I like, is how the current approach amounts
> to refactoring, similar to the cinder move. I feel we should stick to
> that if possible.
>
> John
>
> _______________________________________________ OpenStack-dev
> mailing list OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>