Open Stack

Thanks: TIL.

The filter invocation per host is the bit I was forgetting. 

I'm assuming that the facts about the hosts don't change several times a second so if you held the facts in RAM and then asserted the rules against those facts allowing for age-out/invalidation based on incoming updates then the whole system will run faster. I remember a thread on using dogpile/memoization for this kind of thing.

# Shawn Hartsock

----- Original Message -----
> From: "Yunhong Jiang" <yunhong.jiang at intel.com>
> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
> Sent: Friday, November 1, 2013 1:42:03 PM
> Subject: Re: [openstack-dev] [nova][scheduler]The database access	in	the	scheduler filters
> 
> Shawn, yes, there is 56 VM access every second, and for each VM access, the
> scheduler will invoke filter for each host, that means, for each VM access,
> the filter function will be invoked 10k times.
> So 56 * 10k = 560k, yes, half of 1M, but still big number.
> 
> --jyh
> 
> > -----Original Message-----
> > From: Shawn Hartsock [mailto:hartsocks at vmware.com]
> > Sent: Friday, November 01, 2013 8:20 AM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [nova][scheduler]The database access in the
> > scheduler filters
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Yunhong Jiang" <yunhong.jiang at intel.com>
> > > To: openstack-dev at lists.openstack.org
> > > Sent: Thursday, October 31, 2013 6:39:29 PM
> > > Subject: [openstack-dev] [nova][scheduler]The database access in the
> > 	scheduler filters
> > >
> > > I noticed several filters (AggregateMultiTenancyIsoaltion, ram_filter,
> > > type_filter, AggregateInstanceExtraSpecsFilter) have DB access in the
> > > host_passes(). Some will even access for each invocation.
> > >
> > > Just curios if this is considered a performance issue? With a 10k nodes,
> > 60
> > > VM per node, and 3 hours VM life cycle cloud, it will have more than 1
> > > million DB access per second. Not a small number IMHO.
> > >
> > > Thanks
> > > --jyh
> > >
> > > _______________________________________________
> > > OpenStack-dev mailing list
> > > OpenStack-dev at lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> > 
> > Sorry if I'm dumb, but please try to explain things to me. I don't think I
> > follow...
> > 
> > 10k nodes, 60 VM per node... is 600k VM in the whole cloud. A 3 hour life
> > cycle for a VM means every hour 1/3 the nodes turn over so 200k VM
> > are created/deleted per hour ... divide by 60 for ... 3,333.333 per minute
> > or ... divide by 60 for ... 55.5 VM creations/deletions per second ...
> > 
> > ... did I do that math right? So where's the million DB accesses per second
> > come from? Are the rules fired for every VM on every access so that 600k
> > VM + 1 new VM means the rules fire 600k + 1 times? What? Sorry... really
> > confused.
> > 
> > # Shawn Hartsock
> > 
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>