[OpenStack-Infra] About setting up CI

Wang Shilong wshilong at ddn.com
Tue Feb 28 02:32:44 UTC 2017


Hi,

      I found that it turned out that VM instance could not allocate IP.
Here is some points from here:

       1) I tried to use all in one configurations by devstack, and i had only
one interface with default setting.

[root at r47 ~]# ifconfig
br100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.128.10.70  netmask 255.255.248.0  broadcast 10.128.15.255
        inet6 fe80::fabc:12ff:fe46:9914  prefixlen 64  scopeid 0x20<link>
        ether f8:bc:12:46:99:14  txqueuelen 0  (Ethernet)
        RX packets 850004  bytes 125168729 (119.3 MiB)
        RX errors 0  dropped 57402  overruns 0  frame 0
        TX packets 48066  bytes 28805848 (27.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.24.4.1  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::fce8:8dff:fe51:b34f  prefixlen 64  scopeid 0x20<link>
        inet6 2001:db8::2  prefixlen 64  scopeid 0x0<global>
        ether fe:e8:8d:51:b3:4f  txqueuelen 0  (Ethernet)
        RX packets 41  bytes 5424 (5.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1368 (1.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

em1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::fabc:12ff:fe46:9914  prefixlen 64  scopeid 0x20<link>
        ether f8:bc:12:46:99:14  txqueuelen 1000  (Ethernet)
        RX packets 878780  bytes 142915911 (136.2 MiB)
        RX errors 0  dropped 2289  overruns 0  frame 0
        TX packets 69623  bytes 31466783 (30.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 38  

[root at r47 ~]# brctl show 
bridge name	bridge id		STP enabled	interfaces
br100		8000.f8bc12469914	no		em1
qbr4f716ca2-79		8000.c2387704a265	no		qvb4f716ca2-79
							tap4f716ca2-79
virbr0		8000.525400baf277	yes		virbr0-nic
[root at r47 ~]# ovs-vsctl show
a78c9148-c894-46e2-aaa2-c2e231c51746
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "qr-809690a4-e9"
            tag: 1
            Interface "qr-809690a4-e9"
                type: internal
        Port "qvo4f716ca2-79"
            tag: 2
            Interface "qvo4f716ca2-79"
        Port "tap65aa3db5-d5"
            tag: 1
            Interface "tap65aa3db5-d5"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
        Port "qr-38295378-9b"
            tag: 1
            Interface "qr-38295378-9b"
                type: internal
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "qg-246db997-e5"
            tag: 2
            Interface "qg-246db997-e5"
                type: internal
    Bridge br-ex
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port br-ex
            Interface br-ex
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
    ovs_version: "2.5.0"



------------------------------------------------------
       2) Also tried following conf as Doc said:
[[local|localrc]]
## Neutron options
HOST_IP=10.128.10.70
SERVICE_HOST=10.128.10.70
MYSQL_HOST=10.128.10.70
RABBIT_HOST=10.128.10.70
GLANCE_HOSTPORT=10.128.10.70


# Do not use Nova-Network
disable_service n-net
# Enable Neutron
ENABLED_SERVICES+=,q-svc,q-dhcp,q-meta,q-agt,q-l3

# Neutron options
Q_USE_SECGROUP=True
FLOATING_RANGE="10.128.10.0/24"
IPV4_ADDRS_SAFE_TO_USE="10.0.0.0/22"
Q_FLOATING_ALLOCATION_POOL=start=10.128.10.59,end=10.128.10.60
PUBLIC_NETWORK_GATEWAY="10.128.9.254"
PUBLIC_INTERFACE=em1

Either way, VM instances could not allocate IP, any Ideas?

Thanks,
Shilong

________________________________________
From: Lenny Verkhovsky [lennyb at mellanox.com]
Sent: Thursday, February 09, 2017 16:34
To: Wang Shilong
Cc: Li Xi; Asselin, Ramy; openstack-infra at lists.openstack.org
Subject: RE: About setting up CI

Hi Wang,
Do you see the created machine in the cloud provider?
Can you ping it?
Can you ssh it 'manually'

We had some issues with key's that showed a similar 'ssh' error.
There are also some errors that are not caught/displayed in the logs,
So, sometimes you need to run things manually to get a deeper understanding of the problem

-----Original Message-----
From: Asselin, Ramy [mailto:ramy.asselin at hpe.com]
Sent: Wednesday, February 8, 2017 8:13 PM
To: Wang Shilong <wshilong at ddn.com>; openstack-infra at lists.openstack.org
Cc: Li Xi <lixi at ddn.com>
Subject: Re: [OpenStack-Infra] About setting up CI

Hi Wang,

The ip should be correct. It's possible that the ip in the log message is slightly different because it refers to a different instance floating ip?

If you have a nodepool image uploaded already, I suggest you try to manually launch a node using that image outside of nodepool. You should be able to ssh into it as Ubuntu user using the ssh private key provided when you created the image.

Ramy

-----Original Message-----
From: Wang Shilong [mailto:wshilong at ddn.com]
Sent: Wednesday, February 08, 2017 6:26 AM
To: openstack-infra at lists.openstack.org
Cc: Li Xi <lixi at ddn.com>
Subject: Re: [OpenStack-Infra] About setting up CI

Hello Guys,

       I am following https://github.com/openstack-infra/puppet-openstackci/blob/master/doc/source/third_party_ci.rst
to setup CI.

      I hit following problem from nodepool log.

2017-02-08 22:59:35,529 ERROR nodepool.utils: Exception while testing ssh access to 172.24.4.2:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nodepool/nodeutils.py", line 55, in ssh_connect
    client = SSHClient(ip, username, **connect_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/nodepool/sshclient.py", line 32, in __init__
    allow_agent=allow_agent)
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 305, in connect
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/local/lib/python2.7/dist-packages/paramiko/util.py", line 270, in retry_on_signal
    return function()
  File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 305, in <lambda>
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 110] Connection timed out
2017-02-08 22:59:37,533 ERROR nodepool.NodeLauncher: Timeout launching node id: 4311 in provider: local_01 error: Timeout waiting for ssh access


After some checking, problem is openstack VM instance ip is not what openstack specified, for example for above case, indeed allocation ip of VM is 172.24.4.5.

So nodepool always failed because of this reasons, how can i slove this mismatch problems and proceed CI testing?



Thank you!
Shilong



________________________________________
From: Wang Shilong
Sent: Thursday, November 17, 2016 21:21
To: openstack-infra at lists.openstack.org
Cc: Li Xi
Subject: About setting up CI

Hello folks,

   We are trying to setting up CI testing system for new Lustre cinder Drivers.
I try to do as following URL In centos7:

http://www.joinfu.com/2014/02/setting-up-an-external-openstack-testing-system/

But hit the following problem:

[root at vm7-1 ~]# ./install_master.sh
Using upstream Gerrit user: ddn_openstack Using Jenkins SSH key path: /root/data/jenkins_key
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Error: Could not find template 'openstack_project/puppet.conf.erb' at /root/os-ext-testing/puppet/modules/os_ext_testing/manifests/base.pp:132 on node vm7-1
Error: Could not find template 'openstack_project/puppet.conf.erb' at /root/os-ext-testing/puppet/modules/os_ext_testing/manifests/base.pp:132 on node vm7-1

I guess this Doc might be out of date, could you please give me some hints how I can setup latest CI?

Thanks for your suggestion!
Shilong
_______________________________________________
OpenStack-Infra mailing list
OpenStack-Infra at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

_______________________________________________
OpenStack-Infra mailing list
OpenStack-Infra at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


More information about the OpenStack-Infra mailing list