Open Stack

Well I use a 8 cores 128G RAM physical host :) I did not see much of the
CPU consumption for these 100 containers, so I suspect we can use less
resources.

> -----Message d'origine-----
> De : David Peraza [mailto:david_peraza at persistentsys.com]
> Envoyé : lundi 3 mars 2014 20:27
> À : OpenStack Development Mailing List (not for usage questions)
> Objet : Re: [openstack-dev] [nova] Simulating many fake nova compute
nodes
> for scheduler testing
>
> Thanks Khanh,
>
> I see the potential issue with using threads. Thanks for pointing out.
On using
> containers, that sounds like a cool configuration but that should have a
bigger
> footprint on the host resources than just a separate service instance
like I'm
> doing. I have to admit that 100 fake computes per physical host is good
though.
> How big is your physical host. I'm running a 4 Gig 4 CPU VM. I suspect
your
> physical system is much more equipped.
>
> Regards,
> David Peraza | Openstack Solutions Architect
david_peraza at persistentsys.com |
> Cell: (305)766-2520 Persistent Systems Inc. | Partners in Innovation
> | www.persistentsys.com
>
> -----Original Message-----
> From: Khanh-Toan Tran [mailto:khanh-toan.tran at cloudwatt.com]
> Sent: Tuesday, February 25, 2014 3:49 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [nova] Simulating many fake nova compute
nodes
> for scheduler testing
>
> > > I could do that but I think I need to be able to scale more without
> > > the need to use this much resources. I will like to simulate a cloud
> > > of 100 maybe
> > > 1000 compute nodes that do nothing (Fake driver) this should not
> > > take this much memory. Anyone knows of a more efficient way to
> > > simulate many computes? I was thinking changing the Fake driver to
> > > report many compute services in different threads instead of having
> > > to spawn a process per compute service. Any other ideas?
>
> I'm not sure using threads is a good idea. We need a dedicated resources
pool
> for each compute. If the threads share the same resources pool, then
every new
> VM will change the available resources on all computes, which may lead
to
> unexpected & unpredicted scheduling result. For instance, RamWeigher may
> return the same compute twice instead of spreading, because at each time
it
> finds out that the computes have the same free_ram.
>
> Using compute inside LXC, I created 100 computes per physical host. Here
is
> what I did, it's very simple:
>  -  Creating a LXC with logical volume
>   - Installing a fake nova-compute inside the LXC
>   - Make a booting script that modifies its nova.conf to use its IP
address & starts
> nova-compute
>   - Using the LXC above as the master, clone as many compute as you
like!
>
> (Note that while cloning the LXC, the nova.conf is copied with the
former's IP
> address, that's why we need the booting script.)
>
> Best regards,
>
> Toan
>
>
> > -----Message d'origine-----
> > De : David Peraza [mailto:david_peraza at persistentsys.com]
> > Envoyé : lundi 24 février 2014 21:13
> > À : OpenStack Development Mailing List (not for usage questions) Objet
> > : Re: [openstack-dev] [nova] Simulating many fake nova compute
> nodes
> > for scheduler testing
> >
> > Thanks John,
> >
> > I also think it is a good idea to test the algorithm at unit test
> > level,
> but I will like
> > to try out over amqp as well, that is, we process and threads talking
> > to
> each
> > other over rabbit or qpid. I'm trying to test out performance as well.
> >
> > Regards,
> > David Peraza
> >
> > -----Original Message-----
> > From: John Garbutt [mailto:john at johngarbutt.com]
> > Sent: Monday, February 24, 2014 11:51 AM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [nova] Simulating many fake nova compute
> nodes
> > for scheduler testing
> >
> > On 24 February 2014 16:24, David Peraza
> > <david_peraza at persistentsys.com>
> > wrote:
> > > Hello all,
> > >
> > > I have been trying some new ideas on scheduler and I think I'm
> > > reaching a resource issue. I'm running 6 compute service right on my
> > > 4 CPU 4 Gig VM, and I started to get some memory allocation issues.
> > > Keystone and Nova are already complaining there is not enough
memory.
> > > The obvious solution to add more candidates is to get another VM and
> set
> > another 6 Fake compute service.
> > > I could do that but I think I need to be able to scale more without
> > > the need to use this much resources. I will like to simulate a cloud
> > > of 100 maybe
> > > 1000 compute nodes that do nothing (Fake driver) this should not
> > > take this much memory. Anyone knows of a more efficient way to
> > > simulate many computes? I was thinking changing the Fake driver to
> > > report many compute services in different threads instead of having
> > > to spawn a process per compute service. Any other ideas?
> >
> > It depends what you want to test, but I was able to look at tuning the
> filters and
> > weights using the test at the end of this file:
> >
>
https://review.openstack.org/#/c/67855/33/nova/tests/scheduler/test_cachin
> g
> > _scheduler.py
> >
> > Cheers,
> > John
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > DISCLAIMER
> > ==========
> > This e-mail may contain privileged and confidential information which
> > is
> the
> > property of Persistent Systems Ltd. It is intended only for the use of
> the
> > individual or entity to which it is addressed. If you are not the
> intended recipient,
> > you are not authorized to read, retain, copy, print, distribute or use
> this
> > message. If you have received this communication in error, please
> > notify
> the
> > sender and delete all copies of this message. Persistent Systems Ltd.
> does not
> > accept any liability for virus infected mails.
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
the
> property of Persistent Systems Ltd. It is intended only for the use of
the
> individual or entity to which it is addressed. If you are not the
intended recipient,
> you are not authorized to read, retain, copy, print, distribute or use
this
> message. If you have received this communication in error, please notify
the
> sender and delete all copies of this message. Persistent Systems Ltd.
does not
> accept any liability for virus infected mails.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev