Open Stack

Hi,

I’m not expert in spine-leaf topology TBH but I know that for L3HA neutron creates for each tenant, “tenant network” which usually is vxlan or gre tunnels network. And this works like any other vxlan network created in neutron. So tunnels are established between nodes using L3 and it transports “tenant L2” inside vxlan packets, right? In the same way works this network created for L3 HA needs. And it transports VRRP packets inside this tunnel network (which often is vxlan network).

> On 20 May 2019, at 09:33, Yi Yang (杨燚)-云服务集团 <yangyi01 at inspur.com> wrote:
> 
> Hi, Slawomir, do you mean VRRP over VXLAN? I mean servers in leaf switch are attached to the leaf switch by VLAN and servers handle VxLAN encap and decap, for such case, how can leaf-spine transport a L2 packet to another server in another leaf switch?
> 
> -----邮件原件-----
> 发件人: Slawomir Kaplonski [mailto:skaplons at redhat.com] 
> 发送时间: 2019年5月20日 15:13
> 收件人: Yi Yang (杨燚)-云服务集团 <yangyi01 at inspur.com>
> 抄送: haleyb.dev at gmail.com; openstack-discuss at lists.openstack.org
> 主题: Re: [DVR config] Can we use drv_snat agent_mode in every compute node?
> 重要性: 高
> 
> Hi,
> 
>> On 20 May 2019, at 02:07, Yi Yang (杨燚)-云服务集团 <yangyi01 at inspur.com> wrote:
>> 
>> Brian, thank for your reply. So if I configure 3 compute nodes of many compute node as drv_snat, it doesn't have substantial difference from the case that I configure 3 single network nodes as snat gateway except deployment difference, right? Another question, it doesn't use HA even if we have multiple dvr_snat nodes, right? If enable l3_ha, I think one external router will be scheduled in multiple (2 at least) dvr_snat nodes, for that case, IPs of these HA routers for this one router are same one and are activated by VRRP, right? For l3_ha, two or multiple HA l3 nodes must be in the same L2 network because it uses VRRP (keepalived) to share a VIP, right? For that case, how can we make sure VRRP can work well across leaf switches in a L3 leaf-spine network (servers are connected to leaf switch by L2)?
> 
> That is correct what You are saying. In DVR-HA case, SNAT nodes are working in same way like in “standard” L3HA. So it’s active-backup config and keepalived is deciding which node is active.
> Neutron creates “HA network” for tenant to use for keepalived. It can be e.g. vxlan network and that way You will have L2 between such nodes (routers).
> 
>> 
>> -----邮件原件-----
>> 发件人: Brian Haley [mailto:haleyb.dev at gmail.com]
>> 发送时间: 2019年5月17日 22:11
>> 收件人: Yi Yang (杨燚)-云服务集团 <yangyi01 at inspur.com>
>> 抄送: openstack-discuss at lists.openstack.org
>> 主题: Re: 答复: [DVR config] Can we use drv_snat agent_mode in every compute node?
>> 
>> On 5/16/19 8:29 PM, Yi Yang (杨燚)-云服务集团 wrote:
>>> Thanks Brian, your explanation clarified something, but I don't get the answer if we can have multiple compute nodes are configured to dvr_snat, for this case, SNAT IPs are obviously different. Why do we want to use network node if compute node can do everything?
>> 
>> Hi Yi,
>> 
>> There will only be one DVR SNAT IP allocated for a router on the external network, and only one router scheduled using it, so having dvr_snat mode on a compute node doesn't mean that North/South router will be local, only the East/West portion might be.
>> 
>> Typically people choose to place these on separate systems since the requirements of the role are different - network node could have fewer cores and a 10G nic for higher bandwidth, compute node could have lots of cores for instances but maybe a 1G nic.  There's no reason you can't run dvr_snat everywhere, I would just say it's not common.
>> 
>> -Brian
>> 
>> 
>>> -----邮件原件-----
>>> 发件人: Brian Haley [mailto:haleyb.dev at gmail.com]
>>> 发送时间: 2019年5月16日 21:46
>>> 收件人: Yi Yang (杨燚)-云服务集团 <yangyi01 at inspur.com>
>>> 抄送: openstack-discuss at lists.openstack.org
>>> 主题: Re: [DVR config] Can we use drv_snat agent_mode in every compute node?
>>> 
>>> Hi Yi,
>>> 
>>> I'm a little confused by the question, comments inline.
>>> 
>>> On 5/15/19 11:47 PM, Yi Yang (杨燚)-云服务集团 wrote:
>>>> Hi, folks
>>>> 
>>>> I saw somebody discussed distributed SNAT, but finally they didn’t 
>>>> make agreement on how to implement distributed SNAT, my question is 
>>>> can we use dvr_snat agent_mode in compute node? I understand 
>>>> dvr_snat only does snat but doesn’t do east west routing, right? Can 
>>>> we set dvr_snat and dvr in one compute node at the same time? It is 
>>>> equivalent to distributed SNAT if we can set drv_snat in every 
>>>> compute node, isn’t right? I know Opendaylight can do SNAT in 
>>>> compute node in distributed way, but one external router only can run in one compute node.
>>> 
>>> Distributed SNAT is not available in neutron, there was a spec 
>>> proposed recently though, https://review.opendev.org/#/c/658414
>>> 
>>> Regarding the agent_mode setting for L3, only one mode can be set at a time.  Typically 'dvr_snat' is used on network nodes and 'dvr' on compute nodes because it leads to less resource usage (i.e. namespaces).
>>>  The centralized part of the router hosting the default SNAT IP address will only be scheduled to one of the agents in 'dvr_snat' mode.  All the DVR modes can do East/West routing when an instance is scheduled to the node, and two can do North/South - 'dvr_snat' using the default SNAT IP, and 'dvr' using a floating IP.  'dvr_no_external' can only do East/West.
>>> 
>>> Hopefully that clarifies things.
>>> 
>>> -Brian
>>> 
>>>> I also see https://wiki.openstack.org/wiki/Dragonflow is trying to 
>>>> implement distributed SNAT, what are technical road blocks for 
>>>> distributed SNAT in openstack dvr? Do we have any good way to remove 
>>>> these road blocks?
>>>> 
>>>> Thank you in advance and look forward to getting your replies and insights.
>>>> 
>>>> Also attached official drv configuration guide for your reference.
>>>> 
>>>> https://docs.openstack.org/neutron/stein/configuration/l3-agent.html
>>>> 
>>>> |agent_mode|¶
>>>> <https://docs.openstack.org/neutron/stein/configuration/l3-agent.htm
>>>> l
>>>> #
>>>> DEFAULT.agent_mode>
>>>> 
>>>> Type
>>>> 
>>>> string
>>>> 
>>>> Default
>>>> 
>>>> legacy
>>>> 
>>>> Valid Values
>>>> 
>>>> dvr, dvr_snat, legacy, dvr_no_external
>>>> 
>>>> The working mode for the agent. Allowed modes are: ‘legacy’ - this 
>>>> preserves the existing behavior where the L3 agent is deployed on a 
>>>> centralized networking node to provide L3 services like DNAT, and SNAT.
>>>> Use this mode if you do not want to adopt DVR. ‘dvr’ - this mode 
>>>> enables DVR functionality and must be used for an L3 agent that runs 
>>>> on a compute host. ‘dvr_snat’ - this enables centralized SNAT 
>>>> support in conjunction with DVR. This mode must be used for an L3 
>>>> agent running on a centralized node (or in single-host deployments, e.g. devstack).
>>>> ‘dvr_no_external’ - this mode enables only East/West DVR routing 
>>>> functionality for a L3 agent that runs on a compute host, the 
>>>> North/South functionality such as DNAT and SNAT will be provided by 
>>>> the centralized network node that is running in ‘dvr_snat’ mode. 
>>>> This mode should be used when there is no external network 
>>>> connectivity on the compute host.
>>>> 
> 
> —
> Slawek Kaplonski
> Senior software engineer
> Red Hat
> 

— 
Slawek Kaplonski
Senior software engineer
Red Hat