Open Stack

On May 13, 2011, at 12:31 PM, Sandy Walsh wrote:

> (opening this thread up to the mailing list as it's a good conversation)
> 
> Yes, if you look at HostFilter there is a helper method in all drivers to map from InstanceType to whatever type of query format the driver expects. The weights will need a standardized format I suspect. 
> 

How does the HostFilter fit in with the ZoneAwareScheduler.filter_hosts method? Naively, I would like to do something like this:


class ZoneAwareScheduler(driver.Scheduler):
...
  def filter_hosts(self, num, specs):
    driver = host_filter.FlavorFilter()
    name, cooked = driver.instance_type_to_filter(???) # Need instance_type, have specs instead
    return driver.filter_hosts(self.zone_manager, cooked)
    
But that doesn't work, because the FlavorFilter expects instance types, and the ZoneAwareScheduler expects specs. 


> I'm open to suggestions here, but I thought the specs (aka host query) would be an optional argument on the POST /server command (boot) ... if present it would do the more advanced scheduler. Otherwise, it would fall back to the most-common-denominator InstanceType format: ram, disk, bandwidth. 
> 

In our use case, we would like to avoid requiring the end-users to send optional arguments when requesting instances. Instead, we'd prefer to create new instance types that have additional specs. From the user's point of view, they're still just specifying an instance type, but now the scheduler is taking additional info into account (e.g., cpu architecture, presence of GPU) if a certain instance type is requested.

We will still have to write our own custom scheduler code (unless we use the JSON host filter or use the constraint-based scheduler and specify specs using constraints).


> Specs are qualifications of capabilities (ram > 5G, disk > 500G, etc)
> 
> We're hoping to keep capabilities as Key Value pairs since so many users want so many different things. If there is agreement we could move some capabilities into the InstanceType table. 
> 

Key Value pairs sounds good to me, I was asking more to know about where they will reside. Since instance types are stored in the database (rather than, say, specified in the config file and residing in memory) , will their corresponding specs reside in the database as well?  Can we have a separate InstanceTypeSpecs table that has fields like (instance_type_id, key, value) that allows users to specify arbitrary specs associated with site-specific instance types? 


> PS> Side note: we really need to clean up the parameter lists coming from nova.compute.api:create() into the schedulers ... it's *args/**kwargs hell right now. 
> 

It would be helpful for the reader less familiar with the code if these interfaces were firmed up a little. :)


Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin


> ________________________________________
> From: Lorin Hochstein [lorin at isi.edu]
> Sent: Friday, May 13, 2011 11:50 AM
> To: Sandy Walsh
> Cc: dodcs at pandora.east.isi.edu; Rick Harris
> Subject: Re: dist-scheduler-1
> 
> Sandy:
> 
> Cool...
> 
> On the spec* vs. InstanceType topic, I assume that somewhere along the line, the code will need to do a mapping from instance type to spec. Where are you thinking of encoding the information about  capabilities required for each instance type? For example, are you planning on adding something like an InstanceTypeSpecs table that maps types to the capabilities required?
> 
> * Is "spec" short for "specification"? I assume from code context that it refers to the capabilities that are needed to satisfy a request to run an instance, but wasn't 100% sure.
> 
> Lorin
> --
> Lorin Hochstein, Computer Scientist
> USC Information Sciences Institute
> 703.812.3710
> http://www.east.isi.edu/~lorin
> 
> On May 13, 2011, at 9:43 AM, Sandy Walsh wrote:
> 
>> I've fixed those things in my branch. Rick will pull before merge-prop'ing.
>> 
>> Thanks again!
>> -S
>> 
>> 
>> ________________________________________
>> From: Sandy Walsh
>> Sent: Friday, May 13, 2011 9:33 AM
>> To: Lorin Hochstein
>> Cc: dodcs at pandora.east.isi.edu; Rick Harris
>> Subject: RE: dist-scheduler-1
>> 
>> Nice! All correct Lorin.
>> 
>> Rick just added some more tests. We'll fix that up before issuing the merge prop today.
>> 
>> The whole spec vs. InstanceType thing is something we're still working through. We don't want to bust existing stuff but still need to add the, more flexible, query mechanism. That'll all emerge over the next couple of branches.
>> 
>> Our next step will be getting HostFilter and Rick's CostScheduler tied in (via two parallel merge props)
>> 
>> Great to have your input on this!
>> 
>> -S
>> ________________________________________
>> From: Lorin Hochstein [lorin at isi.edu]
>> Sent: Friday, May 13, 2011 1:12 AM
>> To: Sandy Walsh
>> Cc: dodcs at pandora.east.isi.edu
>> Subject: dist-scheduler-1
>> 
>> Sandy:
>> 
>> I've been playing with your dist-scheduler-1 branch, to see if I can use what's there to do a quick proof-of-concept of heterogeneous nodes using it. (I'm starting off by seeing if I can just get a dead-simple zone scheduler working).
>> 
>> I realize it's still in active development, but I wanted to let you know about the following issues I saw in the code that didn't have TODO associated:
>> 
>> 1.In zone_aware_scheduler.py:
>> 
>> ZoneAwareScheduler.schedule_run_instance takes specs=None as the default, but this will barf when it hits  the "if 'blob' in specs" in the method body. Maybe specs={} instead?
>> 
>> 2. ZoneAwareScheduler.weigh_hosts has the following docstring:
>> 
>>       """Derived classes must override this method and return
>>          a lists of hosts in [(weight, hostname)] format."""
>> 
>> However, based on other parts of the code, it seems thatthis function should return a list of dicts [{'weight': w, 'hostname':h}] instead of a list of pairs.
>> 
>> I created a branch at lp:~lorinh/nova/dist-sched-1 that I'm hacking away at.
>> 
>> Lorin
>> --
>> Lorin Hochstein, Computer Scientist
>> USC Information Sciences Institute
>> 703.812.3710
>> http://www.east.isi.edu/~lorin
>> 
> 
>