Open Stack

Yes, there is an unfortunate possibility of confusion around the word
'subnet'. To some, that means a network segment, or VLAN, and to some
that just means the specific IP subnet. Neutron takes the latter view,
and treats multiple subnets as multiple IP ranges on the same VLAN
(which Neutron calls a 'network').

So if you add multiple subnets to a network, then it is assumed that
means multiple subnets on one physical network, or VLAN (even when
using VXLAN or GRE, there is an internal VLAN segment ID).

There is ongoing work to address the use case where you have a set of
networks that belongs to one tenant, but they are different subnets on
different VLANs with routing between them. See the 'routed networks'
work by Carl Baldwin et. al. That seems to be what the OP was going for
by adding multiple subnets to one network.

Alternately, the desired end state can probably be achieved by simply
using one subnet per network, but using a separate network for each
192.168.10.x, 192.168.11.x, and 192.168.12.x.

-- 
Dan Sneddon         |  Principal OpenStack Engineer
dsneddon at redhat.com |  redhat.com/openstack
650.254.4025        |  dsneddon:irc   @dxs:twitter

On 04/22/2016 05:49 AM, Kevin Benton wrote:
> But if the subnets are on the same VLAN it should work. That's what
> that patch is designed for. There is (currently) an assumption in
> Neutron that subnets in the same network are on the same L2 segment. So
> if a VM wants to communicate with a VM in another subnet on the same
> network, it can cut out the router and use the direct route by ARPing
> for the VM on its own.
> 
> The default is for VMs to only get an IP from one of the subnets on a
> multi subnet network. So if I understand your topology, it exactly
> matches what that patch intended to address (eliminate the hairpinning
> via the router for same network communication).
> 
> Does your router do something special that skipping it breaks things?
> 
> On Apr 22, 2016 05:19, "Remco" <remconl87 at gmail.com
> <mailto:remconl87 at gmail.com>> wrote:
> 
>     Hi Neil,
> 
>     Well that explains...
>     My networking setup does not conform to the commit message in such
>     a way that there are multiple subnets on the network, however the
>     instance does not have an IP address in all of them but only in one.
>     I guess the only option for now is to patch this code, as a
>     workaround.The reason for having multiple subnets on the same L2
>     segment is a migration scenario. So I guess I'll just patch this as
>     the situation will disappear in the near future as normally subnets
>     are separated by L2 domains (vlans) in our setup.
> 
>     Thanks!
>     Remco
> 
>     On Fri, Apr 22, 2016 at 1:16 PM, Neil Jerram
>     <Neil.Jerram at metaswitch.com <mailto:Neil.Jerram at metaswitch.com>> wrote:
> 
>         On 22/04/16 12:03, Remco wrote:
>         > Hi Neil,
>         >
>         > Thanks.
>         > The ip route output is as following, i guess the 0.0.0.0 gateway is only
>         > listed by cloud-init:
>         >
>         > debian at instance:/$ ip route
>         > default via 192.168.10.1 dev eth0
>         > 192.168.10.0/24 <http://192.168.10.0/24>
>         <http://192.168.10.0/24> dev eth0  scope link
>         > 169.254.169.254 via 192.168.10.1 dev eth0
>         > 192.168.11.0/24 <http://192.168.11.0/24>
>         <http://192.168.11.0/24> dev eth0  scope link
>         > 192.168.12.0/24 <http://192.168.12.0/24>
>         <http://192.168.12.0/24> dev eth0  proto kernel  scope
>         > link  src 192.168.10.2
>         >
>         > (ip addresses are altered for security reasons).
> 
>         Thanks for these clarifications.
> 
>         > I'm not sure what creates these routes. I have two suspects: cloud-init
>         > and DHCP. As the same issue is observed on instances without cloud-init
>         > this rules out cloud-init.
>         > We see the same issue on both Windows and Linux instances.
> 
>         OK, I think you're seeing the effect of this DHCP agent code, from
>         neutron/agent/linux/dhcp.py:
> 
>                          host_routes.extend(["%s,0.0.0.0" % (s.cidr)
>         for s in
>                                              self.network.subnets
>                                              if (s.ip_version == 4 and
>                                                  s.cidr != subnet.cidr)])
> 
>         AFAICS there is no obvious knob for suppressing this logic.
> 
>         The code was added in commit 6dce817c7c2, and the commit
>         message says:
> 
>         =============8<=================
>         Provide routes for neighbor IPv4 subnets
> 
>         Network may contain several Subnets.  In this case all these
>         subnets are
>         accessible via same link and two VMs with addresses from different
>         Subnets may talk to each other directly, bypassing default
>         router.  For
>         this to work, "neighbour Subnets" should have entry in VM's routing
>         tables.  RFC3442 describes this situation:
> 
>             In some cases more than one IP subnet may be configured on
>         a link.
>             In such cases, a host whose IP address is in one IP subnet
>         in the
>             link could communicate directly with a host whose IP
>         address is in a
>             different IP subnet on the same link. In cases where a
>         client is
>             being assigned an IP address on an IP subnet on such a
>         link, for each
>             IP subnet in the link other than the IP subnet on which the
>         client
>             has been assigned the DHCP server MAY be configured to
>         specify a
>             router IP address of 0.0.0.0.
> 
>         When network contains more that one IPv4 subnets, report these
>         subnets
>         in 'classless-static-routes' DHCP option.
> 
>         DocImpact
> 
>         Change-Id: Ifcf1d99e0f0136bf52b8d13675b7ccfd48005fab
>         Closes-Bug: #1372885
>         =============8<=================
> 
>         So I guess the next question is: in what way does your
>         networking setup
>         not conform to the assumptions in that commit message?
> 
>         Regards,
>                 Neil
> 
> 
> 
>     _______________________________________________
>     OpenStack-operators mailing list
>     OpenStack-operators at lists.openstack.org
>     <mailto:OpenStack-operators at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 
> 
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>