Open Stack

----- Original Message -----
> From: "Jay Pipes" <jaypipes at gmail.com>
> To: openstack-dev at lists.openstack.org
> Sent: Wednesday, March 4, 2015 9:22:43 PM
> Subject: Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler
> 
> On 03/04/2015 01:51 AM, Attila Fazekas wrote:
> > Hi,
> >
> > I wonder what is the planned future of the scheduling.
> >
> > The scheduler does a lot of high field number query,
> > which is CPU expensive when you are using sqlalchemy-orm.
> > Does anyone tried to switch those operations to sqlalchemy-core ?
> 
> Actually, the scheduler does virtually no SQLAlchemy ORM queries. Almost
> all database access is serialized from the nova-scheduler through the
> nova-conductor service via the nova.objects remoting framework.
> 

It does not helps you.

> > The scheduler does lot of thing in the application, like filtering
> > what can be done on the DB level more efficiently. Why it is not done
> > on the DB side ?
> 
> That's a pretty big generalization. Many filters (check out NUMA
> configuration, host aggregate extra_specs matching, any of the JSON
> filters, etc) don't lend themselves to SQL column-based sorting and
> filtering.
> 

What a basic SQL query can do,
and what is the limit of the SQL is two different thing.
Even if you do not move everything to the DB side,
the dataset the application need to deal with could be limited.

> > There are use cases when the scheduler would need to know even more data,
> > Is there a plan for keeping `everything` in all schedulers process memory
> > up-to-date ?
> > (Maybe zookeeper)
> 
> Zookeeper has nothing to do with scheduling decisions -- only whether or
> not a compute node's "service descriptor" is active or not. The end goal
> (after splitting the Nova scheduler out into Gantt hopefully at the
> start of the L release cycle) is to have the Gantt database be more
> optimized to contain the resource usage amounts of all resources
> consumed in the entire cloud, and to use partitioning/sharding to scale
> the scheduler subsystem, instead of having each scheduler process handle
> requests for all resources in the cloud (or cell...)
> 
What is the current optional usage of zookeeper, 
and what it could be used for is very different thing.
The resource tracking is possible. 

> > The opposite way would be to move most operation into the DB side,
> > since the DB already knows everything.
> > (stored procedures ?)
> 
> See above. This assumes that the data the scheduler is iterating over is
> well-structured and consistent, and that is a false assumption.

With stored procedures you can do almost anything,
and in many ceases it is more readable than an complex query.

> 
> Best,
> -jay
> 
> > Best Regards,
> > Attila
> >
> >
> > ----- Original Message -----
> >> From: "Rui Chen" <chenrui.momo at gmail.com>
> >> To: "OpenStack Development Mailing List (not for usage questions)"
> >> <openstack-dev at lists.openstack.org>
> >> Sent: Wednesday, March 4, 2015 4:51:07 AM
> >> Subject: [openstack-dev] [nova] blueprint about multiple workers supported
> >> 	in nova-scheduler
> >>
> >> Hi all,
> >>
> >> I want to make it easy to launch a bunch of scheduler processes on a host,
> >> multiple scheduler workers will make use of multiple processors of host
> >> and
> >> enhance the performance of nova-scheduler.
> >>
> >> I had registered a blueprint and commit a patch to implement it.
> >> https://blueprints.launchpad.net/nova/+spec/scheduler-multiple-workers-support
> >>
> >> This patch had applied in our performance environment and pass some test
> >> cases, like: concurrent booting multiple instances, currently we didn't
> >> find
> >> inconsistent issue.
> >>
> >> IMO, nova-scheduler should been scaled horizontally on easily way, the
> >> multiple workers should been supported as an out of box feature.
> >>
> >> Please feel free to discuss this feature, thanks.
> >>
> >> Best Regards
> >>
> >>
> >> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>