Open Stack

> > I could do that but I think I need to be able to scale more without
> > the need to use this much resources. I will like to simulate a cloud
> > of 100 maybe
> > 1000 compute nodes that do nothing (Fake driver) this should not take
> > this much memory. Anyone knows of a more efficient way to  simulate
> > many computes? I was thinking changing the Fake driver to report many
> > compute services in different threads instead of having to spawn a
> > process per compute service. Any other ideas?

I'm not sure using threads is a good idea. We need a dedicated resources
pool for each compute. If the threads share the same resources pool, then
every new VM will change the available resources on all computes, which
may lead to unexpected & unpredicted scheduling result. For instance,
RamWeigher may return the same compute twice instead of spreading, because
at each time it finds out that the computes have the same free_ram.

Using compute inside LXC, I created 100 computes per physical host. Here
is what I did, it's very simple:
 -  Creating a LXC with logical volume
  - Installing a fake nova-compute inside the LXC
  - Make a booting script that modifies its nova.conf to use its IP
address & starts nova-compute
  - Using the LXC above as the master, clone as many compute as you like!

(Note that while cloning the LXC, the nova.conf is copied with the
former's IP address, that's why we need the booting script.)

Best regards,

Toan


> -----Message d'origine-----
> De : David Peraza [mailto:david_peraza at persistentsys.com]
> Envoyé : lundi 24 février 2014 21:13
> À : OpenStack Development Mailing List (not for usage questions)
> Objet : Re: [openstack-dev] [nova] Simulating many fake nova compute
nodes
> for scheduler testing
>
> Thanks John,
>
> I also think it is a good idea to test the algorithm at unit test level,
but I will like
> to try out over amqp as well, that is, we process and threads talking to
each
> other over rabbit or qpid. I'm trying to test out performance as well.
>
> Regards,
> David Peraza
>
> -----Original Message-----
> From: John Garbutt [mailto:john at johngarbutt.com]
> Sent: Monday, February 24, 2014 11:51 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [nova] Simulating many fake nova compute
nodes
> for scheduler testing
>
> On 24 February 2014 16:24, David Peraza <david_peraza at persistentsys.com>
> wrote:
> > Hello all,
> >
> > I have been trying some new ideas on scheduler and I think I'm
> > reaching a resource issue. I'm running 6 compute service right on my 4
> > CPU 4 Gig VM, and I started to get some memory allocation issues.
> > Keystone and Nova are already complaining there is not enough memory.
> > The obvious solution to add more candidates is to get another VM and
set
> another 6 Fake compute service.
> > I could do that but I think I need to be able to scale more without
> > the need to use this much resources. I will like to simulate a cloud
> > of 100 maybe
> > 1000 compute nodes that do nothing (Fake driver) this should not take
> > this much memory. Anyone knows of a more efficient way to  simulate
> > many computes? I was thinking changing the Fake driver to report many
> > compute services in different threads instead of having to spawn a
> > process per compute service. Any other ideas?
>
> It depends what you want to test, but I was able to look at tuning the
filters and
> weights using the test at the end of this file:
>
https://review.openstack.org/#/c/67855/33/nova/tests/scheduler/test_cachin
g
> _scheduler.py
>
> Cheers,
> John
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
the
> property of Persistent Systems Ltd. It is intended only for the use of
the
> individual or entity to which it is addressed. If you are not the
intended recipient,
> you are not authorized to read, retain, copy, print, distribute or use
this
> message. If you have received this communication in error, please notify
the
> sender and delete all copies of this message. Persistent Systems Ltd.
does not
> accept any liability for virus infected mails.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev