[Openstack] How to utilize Neutron independently with veths
duhongwei
duhongwei at qiniu.com
Wed May 24 07:25:11 UTC 2017
Dmitry Sutyagin also replied, quote:
Afaik, iptables are set by Nova, and the driver is set via firewall_driver option in nova.conf
After some investigation, I am thinking both Nova and Neutron provide security_group implementation, while Nova operates on vm granularity and Neuton operates on port.
In my openstack environment, I see firewall_driver = nova.virt.firewall.NoopFirewallDriver in nova.conf and firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver in ml2_conf.ini. So it seems I'm using Neutron's security_group implementation.
Back to my question, I tried to name qbr, qvo, qvb as qbr1234567890, ... and, container side veth as veth0, qbr side veth as tap1234567890. As a result, no iptables rules added, therefore packets from veth0 are dropped on qbr1234567890.
So, how would Neutron know that it has to add iptables rules for a device, by device name prefix or what? What can I do to let it happen?
Regards,
Dastan
------------------ Original ------------------
From: "Kevin Benton"<kevin at benton.pub>;
Date: Wed, May 24, 2017 05:21 AM
To: "duhongwei"<duhongwei at qiniu.com>;
Cc: "openstack"<openstack at lists.openstack.org>; "Vallachorum Tyranorum"<ardeleandanflorin at gmail.com>;
Subject: Re: [Openstack] How to utilize Neutron independently with veths
Neutron sets up the iptables rules if you have security groups enabled and the agent firewall is set to iptables_hybrid or neutron.agent.linux.iptables_firewall:OVSHybridIptablesFirewallDriver .
What are you naming your vNIC? The iptables rules setup by the agent match specifically on 'tap' + port UUID prefix. So if the bridge is qvb1234567890 then the vnic you plug into it needs to be named tap1234567890 .
On Tue, May 23, 2017 at 12:15 AM, duhongwei <duhongwei at qiniu.com> wrote:
Thanks Kevin! I've made a big step forward!
Till now, I've successfully connect vNIC directly into br-int without qbr, qvo, and qvb. And, it works well.
However, following your scripts (connect vNIC into qbr, then connect qbr into br-int) exposes another problem. In this scenario, qbr won't forward packets from vNIC to br-int (packets seem to be dropped on qbr).
After some troubleshooting, it turns out to be iptables who drops packets on qbr. Reviewing the FORWARD chain in filter table, packets come from vNIC won't match any rule of neutron-filter-top and neutron-openvswi-FORWARD so that the default policy DROP applies.
So, after setting up all these qbr, qvo, qvb, vNIC, it seems there're still some iptables rules missed. Question is,
Who's adding this iptables rules? (Nova or Neutron?) How can I make it happen?
Regards,
Dastan
------------------ Original ------------------
From: "Kevin Benton"<kevin at benton.pub>;
Date: Mon, May 22, 2017 10:47 PM
To: "duhongwei"<duhongwei at qiniu.com>;
Cc: "openstack"<openstack at lists.openstack.org>; "Vallachorum Tyranorum"<ardeleandanflorin at gmail.com>;
Subject: Re: [Openstack] How to utilize Neutron independently with veths
Yes, the only thing that needs to use the correct MAC is whatever is actually sending traffic.
On May 21, 2017 22:06, "duhongwei" <duhongwei at qiniu.com> wrote:
Thanks for your patient, Kevin.
So qvo could be any veth whose mac address doesn't matter, but veth/tap must have exact the same mac address as port, otherwise it will be anti-spoofed.
qvo's attributes (external-ids) tell neutron which logical port qvo is connecting, so neutron knows how to add flows to ovs br-int and br-tun.
Am I correct?
Regards,
Dastan
------------------ Original ------------------
From: "Kevin Benton"<kevin at benton.pub>;
Date: Sat, May 20, 2017 03:26 AM
To: "duhongwei"<duhongwei at qiniu.com>;
Cc: "openstack"<openstack at lists.openstack.org>; "Vallachorum Tyranorum"<ardeleandanflorin at gmail.com>;
Subject: Re: [Openstack] How to utilize Neutron independently with veths
>After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?
Yes, this last step of creating a veth/tap is missing from my script because I didn't need actual dataplane communication for the tests I was doing.
>1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?
Yeah, if you attach something to qbr to behave like the VM interface, you will need it to be using the mac address of the neutron port, or else the neutron anti-spoofing rules will prevent it from communicating.
>2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?
Yes, the agent will receive to the new port event from ovs, retrieve port details from the server and then setup the flows.
On Fri, May 19, 2017 at 12:09 AM, duhongwei <duhongwei at qiniu.com> wrote:
This script seems easy and cool!
So first we have to create a logical neutron port, then create qbr, qvo and qvb, and plug qvb into qbr, finally plug qvo into ovs br-int. After all these, we create veth/tap (as vm/containers vNIC) and plugin it into qbr then we're able to talk with other vms/containers on the same network through veth/tap, am I understanding it right?
Questions,
1) isn't it necessary that veth/tap's mac address same as neutron port's mac address?
2) after we plug qvo into ovs br-int, neutron just automatically add flows into ovs bridge?
Regards,
Dastan
------------------ Original ------------------
From: "Kevin Benton"<kevin at benton.pub>;
Date: Sat, May 13, 2017 07:46 AM
To: "duhongwei"<duhongwei at qiniu.com>;
Cc: "openstack"<openstack at lists.openstack.org>; "Vallachorum Tyranorum"<ardeleandanflorin at gmail.com>;
Subject: Re: [Openstack] How to utilize Neutron independently with veths
Nova is only responsible for creating the interface and plugging it into the OVS bridge. It's the neutron agent (or alternative neutron backend like OVN) responsible for setting up all of the flows.
Here is a hacky script that I had used to create and delete a bunch of ports like Nova would that you can probably start with: http://paste.openstack.org/show/609478/
On Fri, May 12, 2017 at 4:25 AM, duhongwei <duhongwei at qiniu.com> wrote:
Thanks Kevin!
I'll dig into neutron.agent.linux.interface to see how it works. Before that, would you give me any previews about what steps should be taken to add a veth to a existed Neutron network?
Furthermore, is it Neutron who add a veth to ovs bridge or is it the Neutron caller? (such as Nova)
Who's adding flows to ovs bridge? Neutron or caller?
Regards,
Dastan
------------------ Original ------------------
From: "Kevin Benton"<kevin at benton.pub>;
Date: Fri, May 12, 2017 10:45 AM
To: "duhongwei"<duhongwei at qiniu.com>;
Cc: "openstack"<openstack at lists.openstack.org>; "Vallachorum Tyranorum"<ardeleandanflorin at gmail.com>;
Subject: Re: [Openstack] How to utilize Neutron independently with veths
You want to look in neutron.agent.linux.interface to see how things are plugged into OVS. That's the module used by the L3 agent to plug into OVS/linux bridge/etc.
There is a well defined interface name format corresponding to the port ID and the port ID, Mac address, and a couple of other things I can't recall are set in ovsdb to help the agent identify the port as something it should care about.
On May 9, 2017 04:49, "duhongwei" <duhongwei at qiniu.com> wrote:
Thanks Dan!
I have to write a customized CNI plugin for our product, so it's better if I know more operation details about how to interact with Neutron manually (consider myself as Nova). Therefore Kuryr is not my best option right now, but it's cool!
Regards,
Dastan
------------------ Original ------------------
From: "Vallachorum Tyranorum"<ardeleandanflorin at gmail.com>;
Date: Tue, May 9, 2017 04:08 PM
To: "duhongwei"<duhongwei at qiniu.com>; "openstack"<openstack at lists.openstack.org>;
Subject: Re: [Openstack] How to utilize Neutron independently with veths
Hi,
Please take a look at Kuryr. Maybe this is what you are looking for.
Dan.
On Tue, May 9, 2017 at 10:17 AM duhongwei <duhongwei at qiniu.com> wrote:
Hi everyone,
I'm new to OpenStack and currently interested in the Neutron part of it. What I'm seeking is some advice about how to utilize Neutron independently, to build a virtual network, for Docker containers maybe.
Suppose I've already got Neutron and Keystone installed on controller node and compute nodes. I guess the following steps are required to test a virtual network.
1) create a network
2) create a subnet
3) create two pairs of veths (each pair represents a vm)
for each pair of them:
4) create a port for one end of the veth pair (passing veth's mac address as a parameter)
5) attach another end of the veth pair to ovs bridge
6) ping from one veth pair to another
The above is my general idea, don't know if it is correct and don't know the operation details either.
Expecting your suggestions, any links are appreciated.
Regards,
Dastan
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack at lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack at lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170524/72b7815d/attachment.html>
More information about the Openstack
mailing list