Open Stack

My understanding is that in Icehouse you can have only one L3 agent serving a Neutron subnet at a time, but you could have two Neutron nodes with different L3 agents serving a total of 8000 subnets. Each subnet would basically be served by only one of these L3 agents.

The DHCP agents can run in HA mode, with multiple DHCP agents actively serving the same Neutron subnet, if so you wish. In this case, deploying two Neutron nodes with redundant DHCP agents would limit you to 4096 unique Neutron subnets ("dhcp_agents_per_network = 2").

Another option is of course to deploy two Neutron nodes each running DHCP agents that are not redundant, so you could have 8000 different Neutron subnets spread across two Neutron nodes ("dhcp_agents_per_network = 1").

The following two presentations show some of the possible scalability and HA issues with Neutron:

https://www.youtube.com/watch?v=Yj5cyvYctME
https://www.youtube.com/watch?v=AF9r_VQrcJ0

In Juno, a new HA L3 agent will be implemented (DVR, more info in this doc):
https://docs.google.com/presentation/d/1XJY30ZM0K3xz1U4CVWaQuaqBotGadAFM1xl1UuLtbrk/edit#slide=id.p


I hope this clarifies things a little bit.

George




-----Original Message-----
From: Andreas Scheuring [mailto:scheuran at linux.vnet.ibm.com] 
Sent: Friday, September 19, 2014 3:13 AM
To: openstack at lists.openstack.org
Subject: Re: [Openstack] Question about VXLAN support

Thanks for this clarification. 
Just wondering: Do multiple network nodes (router + dhcp) help to exceed
the 4k subnet limitation? In such a scenario, will dhcp + routing be
distributed or is a second network node just something like a
hot-standby?

Thanks!

-- 
Andreas 
(irc: scheuran)


On Thu, 2014-09-18 at 09:47 -0400, George Mihaiescu wrote:
> The VLAN ID is only locally significant to each compute node, so same
> subnet belonging to same tenant could would have different VLAN tags
> on different compute nodes.
> 
> VLAN tag 1 could be used by subnet1 of tenant A on node1 and by
> subnet1 of tenant B on node 2, with no conflicts.
> 
>  
> 
> Both VXLAN and GRE add a 24-bit header, so the maximum number of
> tunnels is 16 million but each compute node can locally implement only
> a maximum of 4096 different neutron subnets.
> 
> What are the chances that you have more than 4096 instances on a
> compute node, each connected to a different neutron subnet?
> 
> What are the chances that you have more than 409 instances on a
> compute node, each connected to 10 different neutron subnets?
> 
>  
> 
> The same limitation applies to the Neutron node (because a tunnel
> endpoint exists there as well), so you cannot have a Neutron node
> where a L3 agent and a DHCP agent serve more than 4096 Neutron
> subnets, but you would hit other limits by then.
> 
>  
> 
> George
> 
>  
> 
>                                    
> ______________________________________________________________________
> From: BYEONG-GI KIM [mailto:kimbyeonggi at gmail.com] 
> Sent: Wednesday, September 17, 2014 10:41 PM
> To: George Mihaiescu; openstack at lists.openstack.org
> Subject: Re: [Openstack] Question about VXLAN support
> 
> 
>  
> 
> Dear George
> 
>  
> 
> 
> Thank you for the reply. 
> 
> 
>  
> 
> 
> I'm a little confused about your reply. 
> 
> 
>  
> 
> 
> Can be the same tag number assigned to different tenant? For example,
> I assume the situation where a subnet 1 assigned tag number 1 and it
> belongs to tenant A, and a subnet b is also assigned tag number 1 and
> it belongs to tenant B. Or, should be the tag number different even if
> subnets belong to different tenant?
> 
> 
>  
> 
> 
> If the later case, the tag number seems much more strictly limited,
> because a tenant can have many subnet. If a subnet has 10 subnets,
> which means 10 tag numbers must be assigned, the openstack only create
> about 400 tenants.
> 
> 
>  
> 
> 
> Is the VXLAN network type in OpenStack really scalable comparing with
> VLAN or GRE? Or does the current OpenStack just provide functionality
> to handle VXLAN header?
> 
> 
>  
> 
> 
> Please let me know good example about VXLAN usage, which can provide
> scalability for multi-tenant on OpenStack. I'd like to know whether
> more than 100000 tenants could be handled by VXLAN on the latest
> OpenStack implementation or not.
> 
> 
>  
> 
> 
> Best regards
> 
> 
>  
> 
> 
> Byeong-Gi KIM
> 
> 
>  
> 
> 2014-09-18 11:20 GMT+09:00 George Mihaiescu <George.Mihaiescu at q9.com>:
> 
> The internal VLAD ID is indeed limited to 4096 but this internal tag
> number is used to isolate different neutron subnets, not tenants. 
> 
> A tenant could create 10 neutron networks each with its own subnet and
> then start 10 instances each attached to a separate net/subnet. If
> these instances would be scheduled on the same compute node then they
> would all use different internal VLAN IDs (locally unique to that
> node).
> 
>  
> 
> Basically, you're right that there is a built-in limitation of 4096
> instances attached to 4096 different Neutron net/subnets on a compute
> node, but it's not realistic to actually start that many instances on
> a compute node.
> 
>  
> 
> George
> 
>  
> 
>  
> 
>                                    
> ______________________________________________________________________
> From: BYEONG-GI KIM [mailto:kimbyeonggi at gmail.com] 
> Sent: Wednesday, September 17, 2014 8:47 PM
> To: openstack at lists.openstack.org
> Subject: [Openstack] Question about VXLAN support
> 
> 
>  
> 
> Hello.
> 
>  
> 
> 
> I have a question about the VXLAN support on OpenStack.
> 
> 
>  
> 
> 
> As far as I know, the OVS operates like the below:
> 
> 
>  
> 
> 
> 1. A tag number is created to identify each tenant, and it is used
> between br-int and br-tun. Furthermore the tag number is identified as
> a VLAN ID (I checked it via tcpdump).
> 
> 
>  
> 
> 
> 2. After the packet arrived at br-tun, it is encapsulated and VNI
> (VXLAN Network Identifier) is attached. The binding information
> between the VLAN ID (tag number) and the VNI is stored in OVSDB. 
> 
> 
>  
> 
> 
> If the operation is correct, it seems that the number of tenants which
> can be created is still limited to about 4000, which is the supported
> range of VLAN, because the tag number is used to identify each tenant
> at the inside of br-int regardless of the supported range of VNI. 
> 
> 
>  
> 
> 
> If more than 5000 tenants are created in a Compute Node, how could be
> these identified after the packet arrived at br-int? In the theory,
> the 4500th tenant should have 4500 tag number but the tag number is
> presented as VLAN ID so that it cannot be assigned over 4096.
> 
> 
>  
> 
> 
> Any advice and comment would really be appreciated.
> 
> 
>  
> 
> 
> Best regards
> 
> 
>  
> 
> 
> Byeong-Gi KIM
> 
> 
>  
> 
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack