[Openstack] Change from GRE to vlan
Akilesh K
akilesh1597 at gmail.com
Mon Mar 23 12:33:25 UTC 2015
Somehow I fee you have not got the idea correct. packets leaving the
instance will be untagged. They can be captured at the tap interface of the
instance. This can be found by 'neutron port-list' and grepping with the
instance id. The port id should math with a tap interface on your compute
node with similar name.
If you do ovs-vsctl show you should be able to see that particular tap
interface having a tag matching the segmentation id of the network it
belongs to.
The packets will be tagged at 'int-br-xxx' / 'phy-br-xxx' and the physical
interface added to the bridge which inturn is mapped to the physnet of the
'demo - net' (refer bridge-mappings. Once the packet enters 'br-xxx' there
is a vlan translation from that matching the segmentation id to a selected
vlan from 'network_vlan_ranges'.
At the network node it takes the exact replica of the above path to the the
router interface attached to the 'demo-net'. I have summarized all these in
the links to the blog I sent in my first mail.
The only reason packets from compute nodes do not reach the network node
occurs when the vlan translation that happens in compute node does not
match with that in the network node. This could occur when your l2 agents
are out of sync with each other and with the neutron-server. The only way
to correct that is to do an 'ovs-cleanup' and restart the l2 and l3 agent
and terminate and relaunch your instances.
In case you need help I can offer to debug the issue remotely.
Thank you,
Ageeleshwar K
On Mon, Mar 23, 2015 at 1:20 PM, Mitchell Chen <mitchell.chen at mediatek.com>
wrote:
> From the following, I do see the internal network (demo-net)’s
> provider:segmentation_id is 2. The IP in that subnet should map to vlan2,
> please correct me if I am wrong. Is there a way to check the packets left
> from an instance is equipped with a correct header to enter its trunk port?
> In the router namespace, I can not capture any packets. Is there a way to
> debug this?
>
>
>
> Thank you for the advice and help,
>
> Mitchell
>
>
>
> root at controller: # neutron net-list
>
>
> +--------------------------------------+----------+------------------------------------------------------+
>
> | id | name |
> subnets |
>
>
> +--------------------------------------+----------+------------------------------------------------------+
>
> | 5496a11e-6671-406b-9eb8-a6460effc853 | demo-net |
> fc31066e-2bca-4ac1-9f9d-b9853c9bc875 192.168.2.0/24 |
>
> | 7c4a2fe1-a100-495b-875a-137ff528033a | ext-net |
> ea04fd80-2f44-48cc-ad17-4a64aef4649e 172.29.105.0/24 |
>
>
> +--------------------------------------+----------+------------------------------------------------------+
>
> root at controller: # neutron net-show 5496a11e-6671-406b-9eb8-a6460effc853
>
> +---------------------------+--------------------------------------+
>
> | Field | Value |
>
> +---------------------------+--------------------------------------+
>
> | admin_state_up | True |
>
> | id | 5496a11e-6671-406b-9eb8-a6460effc853 |
>
> | name | demo-net |
>
> | provider:network_type | vlan |
>
> | provider:physical_network | physnet1 |
>
> | provider:segmentation_id | 2 |
>
> | router:external | False |
>
> | shared | False |
>
> | status | ACTIVE |
>
> | subnets | fc31066e-2bca-4ac1-9f9d-b9853c9bc875 |
>
> | tenant_id | 1bc3de8b19384858a1a7a6395e1845e3 |
>
> +---------------------------+--------------------------------------+
>
>
>
> root at controller:# neutron net-show 7c4a2fe1-a100-495b-875a-137ff528033a
>
> +---------------------------+--------------------------------------+
>
> | Field | Value |
>
> +---------------------------+--------------------------------------+
>
> | admin_state_up | True |
>
> | id | 7c4a2fe1-a100-495b-875a-137ff528033a |
>
> | name | ext-net |
>
> | provider:network_type | vlan |
>
> | provider:physical_network | physnet1 |
>
> | provider:segmentation_id | 1 |
>
> | router:external | True |
>
> | shared | True |
>
> | status | ACTIVE |
>
> | subnets | ea04fd80-2f44-48cc-ad17-4a64aef4649e |
>
> | tenant_id | 7479d4eabeb14b45a7f38269155ec0f5 |
>
> +---------------------------+--------------------------------------+
>
>
>
> root at neutron:# ip netns
>
> qdhcp-5496a11e-6671-406b-9eb8-a6460effc853
>
> qrouter-b5e219a7-a81c-45d9-9009-a6696b008be3
>
>
>
> root at neutron:# ip netns exec qrouter-b5e219a7-a81c-45d9-9009-a6696b008be3
> tcpdump
>
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>
> listening on qr-3fd978e0-45, link-type EN10MB (Ethernet), capture size
> 65535 bytes
>
>
>
> ^C
>
> 0 packets captured
>
> 0 packets received by filter
>
> 0 packets dropped by kernel
>
>
>
>
>
> *From:* Akilesh K [mailto:akilesh1597 at gmail.com]
> *Sent:* Saturday, March 21, 2015 1:23 AM
>
> *To:* Mitchell Chen
> *Cc:* openstack at lists.openstack.org
> *Subject:* Re: [Openstack] Change from GRE to vlan
>
>
>
> vlan id maps to the segmentation-id of the network. Later when you create
> a subnet under a network the subnet is associated to the
> vlan-id(segmentation-id) of its network.
>
> So all you have to do is provide '--provider:segmentation-id' param while
> creating network. I am not sure with the exact param name, though it should
> be something like what I have mentioned. I do not have access to my
> OpenStack, but you can look it up using 'neutron net-create --help'. I am
> not sure if the dashboard allows you to do this, so use the cli. Finally
> make sure you do this as 'admin' user.
>
> Thank you,
>
> Ageeleshwar K
>
>
>
> On Sat, Mar 21, 2015 at 1:16 AM, Mitchell Chen <mitchell.chen at mediatek.com>
> wrote:
>
> Hi All,
>
>
>
> Thanks you, Ageeleshwar, for the response. I changed the ml2
> configuration for vlan from 1 to 8. The log and dump are in the following.
> But, I have a question how the tenant subnet associated with a vlan? I want
> to have my both tenants with 192.168.2.x and 192.168.3.x subnets to
> associate vlan2 and vlan3. How can I do that with vlan?
>
>
>
> Thanks,
>
> Mitchell
>
>
>
> [ml2]
>
> type_drivers = vlan
>
> tenant_network_types = vlan
>
> mechanism_drivers = openvswitch
>
>
>
> [ml2_type_vlan]
>
> network_vlan_ranges = physnet1:1:8
>
>
>
> [ovs]
>
> local_ip = 192.168.1.218
>
> bridge_mappings = physnet1:br-eth1
>
> tenant_network_type = vlan
>
> network_vlan_ranges = physnet1:1:8
>
> integration_bridge = br-int
>
>
>
> In my compute node, :
>
> 2015-03-20 12:14:32.068 1340 DEBUG neutron.agent.linux.utils
> [req-8a174ab2-0a11-4278-afda-bd8a829831f3 None]
>
> Command: ['sudo', '/usr/bin/neutron-rootwrap',
> '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'dump-flows', 'br-int',
> 'table=22']
>
> Exit code: 0
>
> Stdout: 'NXST_FLOW reply (xid=0x4):\n cookie=0x0, duration=169.473s,
> table=22, n_packets=0, n_bytes=0, idle_age=169, priority=0 actions=drop\n'
>
> Stderr: '' execute
> /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:75
>
> 2015-03-20 12:14:32.069 1340 DEBUG
> neutron.plugins.openvswitch.agent.ovs_neutron_agent
> [req-8a174ab2-0a11-4278-afda-bd8a829831f3 None] Agent rpc_loop -
> iteration:84 completed. Processed ports statistics: {'ancillary':
> {'removed': 0, 'added': 0}, 'regular': {'updated': 0, 'added': 0,
> 'removed': 0}}. Elapsed:0.065 rpc_loop
> /usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1388
>
> 2015-03-20 12:14:34.007 1340 DEBUG
> neutron.plugins.openvswitch.agent.ovs_neutron_agent
> [req-8a174ab2-0a11-4278-afda-bd8a829831f3 None] Agent rpc_loop -
> iteration:85 started rpc_loop
> /usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1287
>
> 2015-03-20 12:14:34.008 1340 DEBUG neutron.agent.linux.utils
> [req-8a174ab2-0a11-4278-afda-bd8a829831f3 None] Running command: ['sudo',
> '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl',
> 'dump-flows', 'br-int', 'table=22'] create_process
> /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:48
>
>
>
> root at compute:# ovs-ofctl dump-flows br-eth1
>
> NXST_FLOW reply (xid=0x4):
>
> cookie=0x0, duration=1111.903s, table=0, n_packets=1070, n_bytes=191705,
> idle_age=0, priority=4,in_port=4,dl_vlan=1 actions=mod_vlan_vid:2,NORMAL
>
> cookie=0x0, duration=1113.399s, table=0, n_packets=508, n_bytes=95459,
> idle_age=6, priority=2,in_port=4 actions=drop
>
> cookie=0x0, duration=1114.149s, table=0, n_packets=613, n_bytes=41365,
> idle_age=2, priority=1 actions=NORMAL
>
>
>
> root at neutron:# ovs-ofctl dump-flows br-eth1
>
> NXST_FLOW reply (xid=0x4):
>
> cookie=0x0, duration=71982.785s, table=0, n_packets=10, n_bytes=796,
> idle_age=65534, hard_age=65534, priority=4,in_port=2,dl_vlan=1
> actions=mod_vlan_vid:2,NORMAL
>
> cookie=0x0, duration=75606.253s, table=0, n_packets=29929,
> n_bytes=5592694, idle_age=5, hard_age=65534, priority=2,in_port=2
> actions=drop
>
> cookie=0x0, duration=75607.504s, table=0, n_packets=1, n_bytes=78,
> idle_age=65534, hard_age=65534, priority=1 actions=NORMAL
>
>
>
>
>
>
>
> *From:* Akilesh K [mailto:akilesh1597 at gmail.com]
> *Sent:* Wednesday, March 18, 2015 11:13 PM
> *To:* Mitchell Chen
> *Cc:* openstack at lists.openstack.org
> *Subject:* Re: [Openstack] Change from GRE to vlan
>
>
>
> Apart from type_driver, there are many other setting in 'ml2_conf.ini'.
> Like below
>
> [ml2]
>
> mechanism_drivers=openvswitch
>
> type_drivers=vlan,flat
>
> tenant_network_types=vlan,flat
>
> [ml2_type_flat]
> flat_networks=Extnet
>
> [ml2_type_vlan]
>
> network_vlan_ranges=Intnet1:100:200
>
> [ovs]
>
> bridge_mappings=Intnet1:br-eth1, Extnet:br-ex
>
> In above configuration I am using one flat External network 'Extnet' and One vlan network 'Intnet1'.
> I would suggest read through the documentation given in the config file. It is a good starting point.
> In case you need more info we have created a few blog posts that could be of help.
>
> https://fosskb.wordpress.com/2014/06/19/l2-connectivity-in-openstack-using-openvswitch-mechanism-driver/
> https://fosskb.wordpress.com/2014/09/15/l3-connectivity-using-neutron-l3-agent/
> https://fosskb.wordpress.com/2014/10/18/openstack-juno-on-ubuntu-14-10/
>
> Do write back if you need any further information.
>
> Thank you,
>
> Ageeleshwar K
>
>
>
>
>
> On Thu, Mar 19, 2015 at 11:31 AM, Mitchell Chen <
> mitchell.chen at mediatek.com> wrote:
>
> Hi All,
>
> I have, step by step, followed the OpenStack icehouse installation guide
> to install control, compute and neutron nodes. They all run fine like
> OpenStack document describes. There is a GRE tunneling connection described
> in OpenStack installation guide from compute nodes to a neutron node. We
> currently plan to change the GRE tunneling connection to vlan, but I don't
> see the Openstack installation describes detail for setting up vlan.
>
> I can see the type driver need to be changed from gre to vlan in
> ml2_conf.ini files in all nodes. I also got a HP switch configured vlan
> trunk ports for each computer and neutron systems to replace the current
> flat switch in the physical private connection.
>
> Besides the above mentioned ml2_conf.ini change and physical connection
> change, is there anything else I need to do for setting the vlan connection
> to get the external/internal nets, subnets and the router to work again to
> build instances? I am going to do this work very soon and will appreciate
> advices from your experience.
>
> Thank you,
> Mitchell
>
> ************* Email Confidentiality Notice ********************
> The information contained in this e-mail message (including any
> attachments) may be confidential, proprietary, privileged, or otherwise
> exempt from disclosure under applicable laws. It is intended to be
> conveyed only to the designated recipient(s). Any use, dissemination,
> distribution, printing, retaining or copying of this e-mail (including its
> attachments) by unintended recipient(s) is strictly prohibited and may
> be unlawful. If you are not an intended recipient of this e-mail, or
> believe
> that you have received this e-mail in error, please notify the sender
> immediately (by replying to this e-mail), delete any and all copies of
> this e-mail (including any attachments) from your system, and do not
> disclose the content of this e-mail to any other person. Thank you!
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
>
> ************* Email Confidentiality Notice ********************
>
> The information contained in this e-mail message (including any
>
> attachments) may be confidential, proprietary, privileged, or otherwise
>
> exempt from disclosure under applicable laws. It is intended to be
>
> conveyed only to the designated recipient(s). Any use, dissemination,
>
> distribution, printing, retaining or copying of this e-mail (including its
>
> attachments) by unintended recipient(s) is strictly prohibited and may
>
> be unlawful. If you are not an intended recipient of this e-mail, or believe
>
> that you have received this e-mail in error, please notify the sender
>
> immediately (by replying to this e-mail), delete any and all copies of
>
> this e-mail (including any attachments) from your system, and do not
>
> disclose the content of this e-mail to any other person. Thank you!
>
>
>
> ************* Email Confidentiality Notice ********************
> The information contained in this e-mail message (including any
> attachments) may be confidential, proprietary, privileged, or otherwise
> exempt from disclosure under applicable laws. It is intended to be
> conveyed only to the designated recipient(s). Any use, dissemination,
> distribution, printing, retaining or copying of this e-mail (including its
> attachments) by unintended recipient(s) is strictly prohibited and may
> be unlawful. If you are not an intended recipient of this e-mail, or believe
> that you have received this e-mail in error, please notify the sender
> immediately (by replying to this e-mail), delete any and all copies of
> this e-mail (including any attachments) from your system, and do not
> disclose the content of this e-mail to any other person. Thank you!
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150323/bb5d3c59/attachment.html>
More information about the Openstack
mailing list