Open Stack

On 04/25/2016 11:33 PM, Jaison Peter wrote:

> I have many concerns about the scaling and right choices , since
> openstack is offering lot of choices and flexibility, especially in
> networking side.Our major challenge was choosing between simplicity and
> performance offered by Linux bridge and features and DVR offered by
> OVS.  We decided to go with OVS, though some were suggesting like OVS is
> slow in large deployments. But the distributed L3 agents and bandwidth
> offered by DVR inclined us towards OVS. Is it a better decision?
>
> But one of the major drawback we are seeing with DVR is the public IP
> consumption.If we have 100 clients and 1 VM per client , eventually
> there will be 100 tenants and 100 routers. Since its a public cloud, we
> have to offer public IP for each VM. In DVR mode, fip name space in
> compute will be consuming one public IP and if 100 VMs are running among
> 20 computes, then total 20 public IPs will be used among computes. And a
> router SNAT name space will be created for each tenant router(Total
> 100)  and each of it will be consuming 1 public  IP and so total 100
> public IPs will be consumed by central SNAT name spaces. So total 100 +
> 20 = 120 public IPs will be used by openstack components and  100 will
> be used as floating IPs (1:1 NAT) by VMs. So we need 220 public IPs for
> providing dedicated public IPs for 100 VMs !! Anything wrong with our
> calculation?

Have you also looked at the namespaces created on the compute nodes and 
the IP addresses they get?

>  From our point of  view 120 IPs used by openstack components in our
> case (providing 1:1 NAT for every VM) is wastage of IPs and no any role
> in network traffic. Centrallized SNAT is useful , if the client is
> opting for VPC like in AWS and he is not attaching floating IPs to all
> instances in his VPC.
>
> So is there any option while creating DVR router to avoid creating
> central SNAT name space in controller node ? So that we can save 100
> public IPs in the above scenario.

That certainly would be nice to be able to do.

DVR certainly does help with instance to instance communication scaling 
- not having to go through the CVR network node(s) is a huge win for 
scaling as far as aggregate network performance is concerned.  But if 
instances are not likely to speak much with one another via floating IPs 
- say because instances on different private networks aren't 
communicating with one another and instances on the same private 
networks can speak with one another via their private IPs then that 
scaling doesn't really matter.  Also if those instances will not be 
communicating (much) with other services in your cloud - Swift comes to 
mind.  So too if all the instances will spend most of their networktime 
speaking with the Big Bad Internet (tm) in which case your cloud's 
connection to the Big Bad Internet will be what gates the aggregate.  In 
all those scenarios you may be just as well-off with CVR.

In terms of other things relating to scale, and drifting a bit from DVR, 
if enable_isolated_metadata is set to true (not sure if that is default 
or not) then there will be two metadata proxies launched for each 
network, in addition to the two metadata proxies launched per router, 
and the metadata proxy launched on each compute node on which an 
instance behind a given DVR router resides.  The latter isn't that big a 
deal for scaling, but the former can be.  Each metadata proxy will want 
~40 MB of RSS, so that is 160 MB of RSS spread across your neutron 
network nodes.  Added to that will be another 10 MB of RSS for a pair of 
dnsmasq processes.

Getting back to DVR and bringing OVS along for the ride, one other 
"cost" is that one still must have Linux bridge in the path to implement 
the security group rules.  Depending on the sort of netperf benchmark 
one runs, that "costs" between 5% and 45% performance.  That was 
measured by setting-up the instances, taking the baseline, and then 
manually "rewiring" bypassing the linux bridge.

happy benchmarking,

rick jones