[openstack-dev] [ovs-discuss] [neutron][networking-ovn] OpenStack Newton using OVN as mechanism driver is giving error.

pranab boruah pranabjyotiboruah at gmail.com
Thu Feb 2 11:56:36 UTC 2017


NB db is not listening on the port 6641:
 # ovn-nbctl --db=tcp:192.168.10.10:6641 show
2017-02-02T11:35:10Z|00001|ovsdb_idl|ERR|tcp:192.168.10.10:6641: requested
schema not found
ovn-nbctl: tcp:192.168.10.10:6641: database connection failed (No such file
or directory)

# ovn-sbctl --db=tcp:192.168.10.10:6642 show
Chassis "e8968c35-11db-4b3b-ab24-c20866ac3dc8"
    hostname: "compute-447"
    Encap geneve
        ip: "172.16.10.11"
        options: {csum="true"}
    Encap vxlan
        ip: "172.16.10.11"
        options: {csum="true"}
Chassis "9f20e7bd-5e16-4273-8ce8-96d0e0a12047"
    hostname: "compute-397"
    Encap geneve
        ip: "172.16.10.12"
        options: {csum="true"}
    Encap vxlan
        ip: "172.16.10.12"
        options: {csum="true"}

# ovsdb-client dump unix:/var/run/openvswitch/ovnnb_db.sock Connection
Connection table
_uuid                                external_ids inactivity_probe
is_connected max_backoff other_config status
target
------------------------------------ ------------ ----------------
------------ ----------- ------------ ---------------
-------------------------
c110ffff-5ba2-4125-8a51-3c3db583f267 {}           []
false        []          {}           {state=BACKOFF} "ptcp:
192.168.10.10:6641"

# ovsdb-client dump unix:/var/run/openvswitch/ovnsb_db.sock Connection
Connection table
_uuid                                external_ids inactivity_probe
is_connected max_backoff other_config read_only status
target
------------------------------------ ------------ ----------------
------------ ----------- ------------ --------- ---------------
-------------------------
1beb9cba-cc01-4981-8a67-a8b265326df9 {}           []
false        []          {}           false     {state=BACKOFF} "ptcp:
192.168.10.10:6642"

No change after restarting Controller node and restating Northd as
mentioned here:
https://github.com/openvswitch/ovs/commit/84d0ca5d00fe01b29163236d48fa0f9105687149

Pranab





On 2 February 2017 at 17:00, Numan Siddique <nusiddiq at redhat.com> wrote:

>
>
> On Thu, Feb 2, 2017 at 4:44 PM, pranab boruah <pranabjyotiboruah at gmail.com
> > wrote:
>
>> Numan,
>>
>> Thanks, but, the above commands didn't changed anything. Checked the logs
>> and found this:
>>
>>
>> Could you run the below two commands and see how it goes
>
>  - "ovn-nbctl --db=tcp:192.168.10.10:6641 show" -> See if this command is
> successful or not. If not then ovsdb-server is either not listening on this
> port or it may be possible that you need to add iptables rule to allow this
> port.
>
>  -
> "ovn-
> ​s​
> bctl --db=tcp:192.168.10.10:664
> ​2​
> show"
>  -​
>
>
> ​On the controller node where you are running the ovsdb-server's run
>  - "sudo ovsdb-client dump unix:/var/run/openvswitch/ovnnb_db.sock
> Connection"
> - "sudo ovsdb-client dump unix:/var/run/openvswitch/ovnsb_db.sock
> Connection"
>> If the Connection table is empty, it means ovsdb-server is not listening
> on the tcp ports.
>
> Thanks
> Numan
>
>
> LOG of NB database:
>>
>> 2017-02-02T11:07:12.366Z|00020|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:07:14.869Z|00021|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:07:16.182Z|00022|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:07:16.183Z|00023|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:07:16.184Z|00024|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:09:55.403Z|00025|ovsdb_jsonrpc_server|INFO|ptcp:6642:
>> 192.168.10.10: remote deconfigured
>> 2017-02-02T11:09:55.403Z|00026|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:09:55.403Z|00027|ovsdb_jsonrpc_server|ERR|Dropped 10 log
>> messages in last 176 seconds (most recently, 159 seconds ago) due to
>> excessive rate
>> 2017-02-02T11:09:55.403Z|00028|ovsdb_jsonrpc_server|ERR|ptcp:6641:
>> 192.168.10.10: listen failed: Address already in use
>> 2017-02-02T11:09:55.403Z|00029|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:09:55.403Z|00030|ovsdb_jsonrpc_server|ERR|ptcp:6641:
>> 192.168.10.10: listen failed: Address already in use
>> 2017-02-02T11:09:57.460Z|00031|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:09:57.460Z|00032|ovsdb_jsonrpc_server|ERR|ptcp:6641:
>> 192.168.10.10: listen failed: Address already in use
>> 2017-02-02T11:09:57.460Z|00033|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:09:59.961Z|00034|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:02.464Z|00035|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:04.967Z|00036|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:07.469Z|00037|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:09.972Z|00038|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:12.475Z|00039|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:14.978Z|00040|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>> 2017-02-02T11:10:17.481Z|00041|socket_util|ERR|6641:192.168.10.10: bind:
>> Address already in use
>>
>> -Pranab
>>
>>
>> On 2 February 2017 at 16:36, Numan Siddique <nusiddiq at redhat.com> wrote:
>>
>>>
>>>
>>> On Thu, Feb 2, 2017 at 4:00 PM, pranab boruah <
>>> pranabjyotiboruah at gmail.com> wrote:
>>>
>>>> Numan,
>>>> Details as requested:
>>>>
>>>> ___________
>>>> ml2_conf.ini:|
>>>> ```````````````````
>>>> ovn_nb_connection = tcp:192.168.10.10:6641
>>>> ovn_sb_connection = tcp:192.168.10.10:6642
>>>> ovn_l3_mode = True
>>>> ovn_l3_scheduler = chance
>>>> ovn_l3_admin_net_cidr = 169.254.128.0/30
>>>> ovn_native_dhcp = True
>>>>
>>>> Note: 192.168.10.10 is the management ip of controller. Firewall,
>>>> NetworkManager and SElinux are disabled on all nodes.
>>>> ____________________________
>>>> output of "ps -aef | grep ovsdb" :  |
>>>> `````````````````````````````````````````````````
>>>> root     21220 21219  0 Feb01 ?        00:00:44 ovsdb-server
>>>> /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info
>>>> --remote=punix:/var/run/openvswitch/db.sock
>>>> --private-key=db:Open_vSwitch,SSL,private_key
>>>> --certificate=db:Open_vSwitch,SSL,certificate
>>>> --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir
>>>> --log-file=/var/log/openvswitch/ovsdb-server.log
>>>> --pidfile=/var/run/openvswitch/ovsdb-server.pid --detach --monitor
>>>>
>>>> root     21324 21323  0 Feb01 ?        00:00:03 ovsdb-server --detach
>>>> --monitor -vconsole:off --log-file=/var/log/openvswitch/ovsdb-server-nb.log
>>>> --remote=punix:/var/run/openvswitch/ovnnb_db.sock
>>>> --pidfile=/var/run/openvswitch/ovnnb_db.pid
>>>> --remote=db:OVN_Northbound,NB_Global,connections
>>>> --unixctl=ovnnb_db.ctl --private-key=db:OVN_Northbound,SSL,private_key
>>>> --certificate=db:OVN_Northbound,SSL,certificate
>>>> --ca-cert=db:OVN_Northbound,SSL,ca_cert /etc/openvswitch/ovnnb_db.db
>>>>
>>>> root     21332 21331  0 Feb01 ?        00:00:03 ovsdb-server --detach
>>>> --monitor -vconsole:off --log-file=/var/log/openvswitch/ovsdb-server-sb.log
>>>> --remote=punix:/var/run/openvswitch/ovnsb_db.sock
>>>> --pidfile=/var/run/openvswitch/ovnsb_db.pid
>>>> --remote=db:OVN_Southbound,SB_Global,connections
>>>> --unixctl=ovnsb_db.ctl --private-key=db:OVN_Southbound,SSL,private_key
>>>> --certificate=db:OVN_Southbound,SSL,certificate
>>>> --ca-cert=db:OVN_Southbound,SSL,ca_cert /etc/openvswitch/ovnsb_db.db
>>>>
>>>> Note: NB and SB databases are supposed to be listening on a tcp socket.
>>>> But it isn't.
>>>>
>>>>
>>> ​Running the below commands should solve your problem
>>>   - ovn-nbctl set-connection ptcp:6641:192.168.10.10
>>>
>>> - ovn-
>>> ​s​
>>> bctl set-connection ptcp:664
>>> ​2​
>>> :192.168.10.10
>>>
>>> L
>>> ​et me know if you still face problems.
>>>
>>> The reason for this is this commit -
>>> https://github.com/openvswitch/ovs/commit/84d0ca5d00fe01b291
>>> 63236d48fa0f9105687149
>>>
>>> P
>>> ​lease have a look at it.
>>>
>>> Thanks
>>> Numan
>>>>>>>>>
>>>> _________________________
>>>> output of logs                       |
>>>> ```````````````````````````````````````````
>>>> Neutron server log :
>>>>      - http://paste.openstack.org/show/597320/
>>>> ovs-nothd log:
>>>>      - http://paste.openstack.org/show/597322/
>>>> ovs-vswitchd log on Controller:
>>>>       - http://paste.openstack.org/show/597324/
>>>> _________________________________________
>>>> # ovn-nbctl and ovn-sbctl show no output.           |
>>>> ```````````````````````````````````````````````````````````````````````
>>>> ________________________________________
>>>> OVN RPMS installation and download details:   |
>>>> ``````````````````````````````````````````````````````````````````````
>>>> On controller:
>>>> # git clone https://github.com/openvswitch/ovs.git
>>>> # ./boot.sh
>>>> # ./configure
>>>> # make rpm-fedora RPMBUILD_OPT="--without check"
>>>> # make rpm-fedora-kmod
>>>>
>>>> ALL nodes:
>>>>
>>>> # rpm -i openvswitch-kmod-2.6.90-1.el7.centos.x86_64.rpm#  rpm -U openvswitch-2.6.90-1.el7.centos.x86_64.rpm
>>>>
>>>> Only Controller:
>>>>
>>>> # rpm -i openvswitch-ovn-common-*.x86_64.rpm# rpm -i openvswitch-ovn-central-*.x86_64.rpm
>>>>
>>>> Only Compute:
>>>>
>>>> # rpm -i openvswitch-ovn-common-*.x86_64.rpm# rpm -i openvswitch-ovn-host-*.x86_64.rpm
>>>>
>>>> # lsmod | grep open
>>>>      - shows all the required modules
>>>>
>>>> Let me know if you need anything else.
>>>>
>>>> Thanks.
>>>> pjb
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 2 February 2017 at 15:23, Numan Siddique <nusiddiq at redhat.com>
>>>> wrote:
>>>>
>>>>> Adding openstack-dev ML with [networking-ovn] tag.
>>>>>
>>>>> Please see comments inline
>>>>>
>>>>>
>>>>> On Thu, Feb 2, 2017 at 3:01 PM, pranab boruah <
>>>>> pranabjyotiboruah at gmail.com> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>> We have a OpenStack multi-node lab. Everything was working, until I
>>>>>> tried integrating with OVN.
>>>>>> Followed the official guide for integration:
>>>>>>
>>>>>> http://docs.openstack.org/developer/networking-ovn/install.html
>>>>>>
>>>>>> Lab details:
>>>>>> 1 Controller, 2 computes
>>>>>>
>>>>>> CentOS-7.3, OpenStack Newton, separate network for mgmt and tunnel.
>>>>>>
>>>>>> Error details :
>>>>>> Neutron Server log shows :
>>>>>>
>>>>>>  ERROR networking_ovn.ovsdb.impl_idl_ovn [-] OVS database connection
>>>>>> to OVN_Northbound failed with error: '{u'error': u'unknown database',
>>>>>> u'details': u'get_schema request specifies unknown database
>>>>>> OVN_Northbound', u'syntax': u'["OVN_Northbound"]'}'. Verify that the OVS
>>>>>> and OVN services are available and that the 'ovn_nb_connection' and
>>>>>> 'ovn_sb_connection' configuration options are correct.
>>>>>>
>>>>>> Stuck in this for more than 2 week. Starting to hate my life. Please
>>>>>> help. Before I tried installing in physical machines, followed the same
>>>>>> guide and installed in VMs. Everything was working then.
>>>>>>
>>>>>> Solutions I tried:
>>>>>> 1) Completely fresh installing everything.
>>>>>> 2) Rebuilding ovs
>>>>>> 3) Check SB and NB configuration options in plugin.ini are exactly
>>>>>> correct.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>> Could you please share
>>>>>  - the values of ovn_nb_connection and ovn_sb_connection defined in
>>>>> ml2_conf.ini.
>>>>>  - the output of "ps -aef | grep ovsdb" on the node where you are
>>>>> running the ovn-northd server.
>>>>>  - If you could share the output of neutron-server logs in some paste
>>>>> link, it would be helpful.
>>>>>
>>>>> Are you able to run "ovn-nbctl show" ? I presume you have installed
>>>>> the appropriate ovn RPMS ? If not so, please let me know how you have
>>>>> installed it.
>>>>>
>>>>> Thanks
>>>>> Numan
>>>>>
>>>>>>>>>>
>>>>>
>>>>>> Let me know if additional details are required.
>>>>>>
>>>>>>
>>>>>> -pjb
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> discuss mailing list
>>>>>> discuss at openvswitch.org
>>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170202/47bccbfa/attachment.html>


More information about the OpenStack-dev mailing list