[Openstack] Openstack XCP with OVS Quantum
Rajshree Thorat
rajshree.thorat at gslab.com
Thu Oct 31 09:25:31 UTC 2013
On 10/31/2013 2:53 PM, Rajshree Thorat wrote:
> Hi All,
>
> I have successfully configured Openstack Havana with xen
> hypervisor(XCP). Initially creating/deleting
> instances from OpenStack works as expected but networking part(neutron
> with OpenvSwitch) was not working.
>
> The steps I performed to make it work are as below:
>
> Normal flow to get DHCP IP:
>
> - VM boots and asks for an IP through DHCP
> - The Nova Compute has a GRE tunnel to the OpenStack Networking node
> where the neutron/openvSwitch agent provides an IP to the VM.
>
> VM <---> Nova Compute Node <---> GRE tunnel <---> OpenStack Networking
> node <---> DHCP agent
>
> In case of XCP when guest VM boots it sends a DHCP request to dom0
> through xenapi but dom0 unable to communicate with
> OpenStack Networking node over GRE tunnel.
>
> To allow VM's to communicate with Network node over GRE tunnel, we can
> assign one more nic(eth2) which is part of xapi1 of dom0
> to nova-compute and add eth2 to br-int on nova-compute.
>
> xapi1 is a openstack network bridge in dom0.
>
> Now the packet will traverse as
>
> VM --> xapi1(dom0) --> eth2(compute) --> br-tun(compute) -->
> Network-node(over GRE tunnel)
>
> VM <-- xapi1(dom0) <-- eth2(compute) <-- br-tun(compute) <--
> Network-node(over GRE tunnel)
>
> Inbuilt Openvswitch-controller configures the v-switches to allow only
> specific flows which
> matches the rules installed on them. Even if we add eth2 to br-int, we
> will also need to add
> generic rules to br-tun such that they are able to pass the packets
> received from eth2
> to br-int, then to br-tun and then to network node over GRE tunnel.
> That's it you are done !
>
> dump-flows before adding rules:
>
> root at compute:~# ovs-ofctl dump-flows br-tun
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=3.248s, table=0, n_packets=0, n_bytes=0,
> idle_age=3, priority=1,in_port=1 actions=resubmit(,1)
> cookie=0x0, duration=2.069s, table=0, n_packets=0, n_bytes=0,
> idle_age=2, priority=1,in_port=2 actions=resubmit(,2)
> cookie=0x0, duration=3.187s, table=0, n_packets=1, n_bytes=70,
> idle_age=2, priority=0 actions=drop
> cookie=0x0, duration=3.066s, table=1, n_packets=0, n_bytes=0,
> idle_age=3, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00
> actions=resubmit(,21)
> cookie=0x0, duration=3.126s, table=1, n_packets=0, n_bytes=0,
> idle_age=3, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00
> actions=resubmit(,20)
> cookie=0x0, duration=3.006s, table=2, n_packets=0, n_bytes=0,
> idle_age=3, priority=0 actions=drop
> cookie=0x0, duration=2.946s, table=3, n_packets=0, n_bytes=0,
> idle_age=2, priority=0 actions=drop
> cookie=0x0, duration=2.886s, table=10, n_packets=0, n_bytes=0,
> idle_age=2, priority=1
> actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
> cookie=0x0, duration=2.825s, table=20, n_packets=0, n_bytes=0,
> idle_age=2, priority=0 actions=resubmit(,21)
> cookie=0x0, duration=2.766s, table=21, n_packets=0, n_bytes=0,
> idle_age=2, priority=0 actions=drop
>
> Add flows:
>
> root at compute:~# ovs-vsctl add-port br-int eth2 tag=1
>
> Here the neutron-plugin-openvswitch-agent
> has put port eth2 into VLAN 1 (tag 1) on br-int.
>
> root at compute:~# ovs-ofctl add-flow br-tun
> priority=3,in_port=1,dl_vlan=1,actions=set_tunnel:0x1,NORMAL
>
> It is for outgoing traffic from br-int
> VLAN 1 - it sets the GRE key to 0x1 and gives it the NORMAL action.
>
> root at compute:~# ovs-ofctl add-flow br-tun
> priority=2,tun_id=0x1,actions=mod_vlan_vid:1,NORMAL
>
> This flow accepts incoming traffic.
>
> dump-flows after adding rules:
>
> root at compute:~# ovs-ofctl dump-flows br-tun
> NXST_FLOW reply (xid=0x4):
> cookie=0x0, duration=115.467s, table=0, n_packets=5, n_bytes=958,
> idle_age=58, priority=2,tun_id=0x1 actions=mod_vlan_vid:1,NORMAL
> cookie=0x0, duration=133.203s, table=0, n_packets=5, n_bytes=830,
> idle_age=61, priority=3,in_port=1,dl_vlan=1 actions=set_tunnel:0x1,NORMAL
> cookie=0x0, duration=343.011s, table=0, n_packets=7, n_bytes=1230,
> idle_age=186, priority=1,in_port=1 actions=resubmit(,1)
> cookie=0x0, duration=341.832s, table=0, n_packets=0, n_bytes=0,
> idle_age=341, priority=1,in_port=2 actions=resubmit(,2)
> cookie=0x0, duration=342.95s, table=0, n_packets=4, n_bytes=300,
> idle_age=334, priority=0 actions=drop
> cookie=0x0, duration=342.829s, table=1, n_packets=7, n_bytes=1230,
> idle_age=186, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00
> actions=resubmit(,21)
> cookie=0x0, duration=342.889s, table=1, n_packets=0, n_bytes=0,
> idle_age=342, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00
> actions=resubmit(,20)
> cookie=0x0, duration=342.769s, table=2, n_packets=0, n_bytes=0,
> idle_age=342, priority=0 actions=drop
> cookie=0x0, duration=342.709s, table=3, n_packets=0, n_bytes=0,
> idle_age=342, priority=0 actions=drop
> cookie=0x0, duration=342.649s, table=10, n_packets=0, n_bytes=0,
> idle_age=342, priority=1
> actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1
> cookie=0x0, duration=342.588s, table=20, n_packets=0, n_bytes=0,
> idle_age=342, priority=0 actions=resubmit(,21)
> cookie=0x0, duration=342.529s, table=21, n_packets=7, n_bytes=1230,
> idle_age=186, priority=0 actions=drop
>
> This works for me. Please correct me if I am breaking any
> functionality of neutron.
>
> Regards,
> Rajshree
>
> On 10/4/2013 7:57 PM, Bob Ball wrote:
>>
>> While I'm a big supporter of xenserver-core (which is the name of the
>> packages Endre mentions) -- the packages and their OpenStack
>> integration is currently very new. We're working through a number of
>> issues in the packaging at the moment and will hopefully have a more
>> stable point in a few weeks' time.
>>
>> For the time being I would personally recommend using XenServer 6.2
>> -- which is also free, unlicensed and fully open source.
>>
>> On the other hand, if you chose to use xenserver-core and find
>> issues, please report them on github
>> (https://github.com/xapi-project/xenserver-core/) and we'll get them
>> fixed as soon as we can -- or submit a pull request if you find the
>> fix yourself!
>>
>> Bob
>>
>> *From:*Endre Karlson [mailto:endre.karlson at gmail.com]
>> *Sent:* 04 October 2013 14:57
>> *To:* Rajshree Thorat
>> *Cc:* openstack at ask.openstack.org; openstack Users
>> *Subject:* Re: [Openstack] Openstack XCP with OVS Quantum
>>
>> I'll suggest you to move away from XCP if you can and check out the
>> new work done by Citrix to make XenServer packages / functionality
>> available to be installed on stuff like CentOS 6 and Ubuntu + others.
>>
>> Endre.
>>
>> 2013/10/4 Rajshree Thorat <rajshree.thorat at gslab.com
>> <mailto:rajshree.thorat at gslab.com>>
>>
>> Hi,
>>
>> I am trying to use Openstack Grizzly to control XCP hypervisor with
>> quantum ovs plugin. I tried with the steps which are mentioned in
>> official document
>> (http://docs.openstack.org/grizzly/openstack-compute/install/apt/content/introduction-to-xen.html#xen-config-reference)
>> but it doesn't work for me. I am not able to start nova-compute
>> service. It is giving me following error:
>>
>> 2013-10-04 18:23:44.861 32548 INFO nova.manager [-] Skipping periodic
>> task _periodic_update_dns because its interval is negative
>> 2013-10-04 18:23:44.902 32548 INFO nova.virt.driver [-] Loading
>> compute driver 'xenapi.XenAPIDriver'
>> 2013-10-04 18:23:44.922 32548 CRITICAL nova [-] [Errno 111] ECONNREFUSED
>> 2013-10-04 18:23:44.922 32548 TRACE nova Traceback (most recent call
>> last):
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/bin/nova-compute", line 83, in <module>
>> 2013-10-04 18:23:44.922 32548 TRACE nova db_allowed=False)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/service.py", line 534, in create
>> 2013-10-04 18:23:44.922 32548 TRACE nova db_allowed=db_allowed)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/service.py", line 413, in __init__
>> 2013-10-04 18:23:44.922 32548 TRACE nova self.manager =
>> manager_class(host=self.host, *args, **kwargs)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 353,
>> in __init__
>> 2013-10-04 18:23:44.922 32548 TRACE nova self.driver =
>> driver.load_compute_driver(self.virtapi, compute_driver)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/virt/driver.py", line 931, in
>> load_compute_driver
>> 2013-10-04 18:23:44.922 32548 TRACE nova virtapi)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/importutils.py",
>> line 51, in import_object_ns
>> 2013-10-04 18:23:44.922 32548 TRACE nova return
>> import_class(import_value)(*args, **kwargs)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/driver.py", line
>> 139, in __init__
>> 2013-10-04 18:23:44.922 32548 TRACE nova self._session =
>> XenAPISession(url, username, password, self.virtapi)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/driver.py", line
>> 630, in __init__
>> 2013-10-04 18:23:44.922 32548 TRACE nova url =
>> self._create_first_session(url, user, pw, exception)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/driver.py", line
>> 641, in _create_first_session
>> 2013-10-04 18:23:44.922 32548 TRACE nova
>> session.login_with_password(user, pw)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/local/lib/python2.7/dist-packages/XenAPI.py", line 182, in <lambda>
>> 2013-10-04 18:23:44.922 32548 TRACE nova return lambda *params:
>> self._login(name, params)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/local/lib/python2.7/dist-packages/XenAPI.py", line 148, in _login
>> 2013-10-04 18:23:44.922 32548 TRACE nova result =
>> _parse_result(getattr(self, 'session.%s' % method)(*params))
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
>> 2013-10-04 18:23:44.922 32548 TRACE nova return
>> self.__send(self.__name, args)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
>> 2013-10-04 18:23:44.922 32548 TRACE nova verbose=self.__verbose
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
>> 2013-10-04 18:23:44.922 32548 TRACE nova return
>> self.single_request(host, handler, request_body, verbose)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/xmlrpclib.py", line 1292, in single_request
>> 2013-10-04 18:23:44.922 32548 TRACE nova self.send_content(h,
>> request_body)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/xmlrpclib.py", line 1439, in send_content
>> 2013-10-04 18:23:44.922 32548 TRACE nova
>> connection.endheaders(request_body)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/httplib.py", line 954, in endheaders
>> 2013-10-04 18:23:44.922 32548 TRACE nova self._send_output(message_body)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/httplib.py", line 814, in _send_output
>> 2013-10-04 18:23:44.922 32548 TRACE nova self.send(msg)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/httplib.py", line 776, in send
>> 2013-10-04 18:23:44.922 32548 TRACE nova self.connect()
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/httplib.py", line 757, in connect
>> 2013-10-04 18:23:44.922 32548 TRACE nova self.timeout,
>> self.source_address)
>> 2013-10-04 18:23:44.922 32548 TRACE nova File
>> "/usr/lib/python2.7/dist-packages/eventlet/green/socket.py", line 59,
>> in create_connection
>> 2013-10-04 18:23:44.922 32548 TRACE nova raise error, msg
>> 2013-10-04 18:23:44.922 32548 TRACE nova error: [Errno 111] ECONNREFUSED
>> 2013-10-04 18:23:44.922 32548 TRACE nova
>>
>> It seems network with openvswitch is not configured properly. Is
>> there any OVS quantum plugin requires to run under dom0? Is there
>> any document which can briefly describe how Openstack + XCP works
>> with OVS Quantum?
>>
>> Regards,
>> Rajshree
>>
>> _______________________________________________
>> Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openstack at lists.openstack.org
>> <mailto:openstack at lists.openstack.org>
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131031/6db554a4/attachment.html>
More information about the Openstack
mailing list