Open Stack

Clint,

There are many PROS and CONS in both of approaches.

Reinventing wheel (in this case it's quite simple task) and it gives more
flexibility and doesn't require
usage of ZK/Consul (which will simplify integration of it with current
system)

Using ZK/Consul for POC may save a lot of time and as well we are
delegating part of work
to other communities (which may lead in better supported/working code).

By the way some of the parts (like sync of schedulers) stuck on review in
Nova project.

Basically for POC we can use anything and using ZK/Consul may reduce
resources for development
which is good.

Best regards,
Boris Pavlovic

On Sun, Oct 11, 2015 at 12:23 AM, Clint Byrum <clint at fewbar.com> wrote:

> Excerpts from Boris Pavlovic's message of 2015-10-11 00:02:39 -0700:
> > 2Everybody,
> >
> > Just curios why we need such complexity.
> >
> >
> > Let's take a look from other side:
> > 1) Information about all hosts (even in case of 100k hosts) will be less
> > then 1 GB
> > 2) Usually servers that runs scheduler service have at least 64GB RAM and
> > more on the board
> > 3) math.log(100000) < 12  (binary search per rule)
> > 4) We have less then 20 rules for scheduling
> > 5) Information about hosts is updated every 60 seconds (no updates host
> is
> > dead)
> >
> >
> > According to this information:
> > 1) We can store everything in RAM of single server
> > 2) We can use Python
> > 3) Information about hosts is temporary data and shouldn't be stored in
> > persistence storage
> >
> >
> > Simplest architecture to cover this:
> > 1) Single RPC service that has two methods: find_host(rules),
> > update_host(host, data)
> > 2) Store information about hosts  like a dict (host_name->data)
> > 3) Create for each rule binary tree and update it on each host update
> > 4) Make a algorithm that will use binary trees to find host based on
> rules
> > 5) Each service like compute node, volume node, or neutron will send
> > updates about host
> >    that they managed (cross service scheduling)
> > 6) Make a algorithm that will sync host stats in memory between different
> > schedulers
>
> I'm in, except I think this gets simpler with an intermediary service
> like ZK/Consul to keep track of this 1GB of data and replace the need
> for 6, and changes the implementation of 5 to "updates its record and
> signals its presence".
>
> What you've described is where I'd like to experiment, but I don't want
> to reinvent ZK or Consul or etcd when they already exist and do such a
> splendid job keeping observers informed of small changes in small data
> sets. You still end up with the same in-memory performance, and this is
> in line with some published white papers from Google around their use
> of Chubby, which is their ZK/Consul.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151011/d12b507c/attachment.html>