[OpenStack-Infra] About setting up CI
Lenny Verkhovsky
lennyb at mellanox.com
Tue Feb 28 07:39:26 UTC 2017
Wang,
Did you try to create VM manually in the cloud provider?
You can try setting 'network' in your /etc/nodepool/nodepool.yaml file
providers:
- name: local_01
username: 'admin'
password: 'admin'
auth-url: 'http://example.com:35357/v2.0'
project-name: 'admin'
max-servers: 9
launch-timeout: 600
networks:
- name: private_network
images:
- name: centos7-build
min-ram: 1024
diskimage: centos7-build
username: jenkins
private-key: '/home/nodepool/.ssh/id_rsa'
There is also a weekly Third Party CI Meetings [1] that you can get some answers
[1] https://wiki.openstack.org/wiki/Meetings/ThirdParty
-----Original Message-----
From: Wang Shilong [mailto:wshilong at ddn.com]
Sent: Tuesday, February 28, 2017 4:33 AM
To: Lenny Verkhovsky <lennyb at mellanox.com>
Cc: Li Xi <lixi at ddn.com>; Asselin, Ramy <ramy.asselin at hpe.com>; openstack-infra at lists.openstack.org
Subject: RE: About setting up CI
Hi,
I found that it turned out that VM instance could not allocate IP.
Here is some points from here:
1) I tried to use all in one configurations by devstack, and i had only one interface with default setting.
[root at r47 ~]# ifconfig
br100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.128.10.70 netmask 255.255.248.0 broadcast 10.128.15.255
inet6 fe80::fabc:12ff:fe46:9914 prefixlen 64 scopeid 0x20<link>
ether f8:bc:12:46:99:14 txqueuelen 0 (Ethernet)
RX packets 850004 bytes 125168729 (119.3 MiB)
RX errors 0 dropped 57402 overruns 0 frame 0
TX packets 48066 bytes 28805848 (27.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.24.4.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::fce8:8dff:fe51:b34f prefixlen 64 scopeid 0x20<link>
inet6 2001:db8::2 prefixlen 64 scopeid 0x0<global>
ether fe:e8:8d:51:b3:4f txqueuelen 0 (Ethernet)
RX packets 41 bytes 5424 (5.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16 bytes 1368 (1.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
em1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::fabc:12ff:fe46:9914 prefixlen 64 scopeid 0x20<link>
ether f8:bc:12:46:99:14 txqueuelen 1000 (Ethernet)
RX packets 878780 bytes 142915911 (136.2 MiB)
RX errors 0 dropped 2289 overruns 0 frame 0
TX packets 69623 bytes 31466783 (30.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 38
[root at r47 ~]# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.f8bc12469914 no em1
qbr4f716ca2-79 8000.c2387704a265 no qvb4f716ca2-79
tap4f716ca2-79
virbr0 8000.525400baf277 yes virbr0-nic
[root at r47 ~]# ovs-vsctl show
a78c9148-c894-46e2-aaa2-c2e231c51746
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "qr-809690a4-e9"
tag: 1
Interface "qr-809690a4-e9"
type: internal
Port "qvo4f716ca2-79"
tag: 2
Interface "qvo4f716ca2-79"
Port "tap65aa3db5-d5"
tag: 1
Interface "tap65aa3db5-d5"
type: internal
Port br-int
Interface br-int
type: internal
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port "qr-38295378-9b"
tag: 1
Interface "qr-38295378-9b"
type: internal
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "qg-246db997-e5"
tag: 2
Interface "qg-246db997-e5"
type: internal
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port br-ex
Interface br-ex
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
ovs_version: "2.5.0"
------------------------------------------------------
2) Also tried following conf as Doc said:
[[local|localrc]]
## Neutron options
HOST_IP=10.128.10.70
SERVICE_HOST=10.128.10.70
MYSQL_HOST=10.128.10.70
RABBIT_HOST=10.128.10.70
GLANCE_HOSTPORT=10.128.10.70
# Do not use Nova-Network
disable_service n-net
# Enable Neutron
ENABLED_SERVICES+=,q-svc,q-dhcp,q-meta,q-agt,q-l3
# Neutron options
Q_USE_SECGROUP=True
FLOATING_RANGE="10.128.10.0/24"
IPV4_ADDRS_SAFE_TO_USE="10.0.0.0/22"
Q_FLOATING_ALLOCATION_POOL=start=10.128.10.59,end=10.128.10.60
PUBLIC_NETWORK_GATEWAY="10.128.9.254"
PUBLIC_INTERFACE=em1
Either way, VM instances could not allocate IP, any Ideas?
Thanks,
Shilong
________________________________________
From: Lenny Verkhovsky [lennyb at mellanox.com]
Sent: Thursday, February 09, 2017 16:34
To: Wang Shilong
Cc: Li Xi; Asselin, Ramy; openstack-infra at lists.openstack.org
Subject: RE: About setting up CI
Hi Wang,
Do you see the created machine in the cloud provider?
Can you ping it?
Can you ssh it 'manually'
We had some issues with key's that showed a similar 'ssh' error.
There are also some errors that are not caught/displayed in the logs, So, sometimes you need to run things manually to get a deeper understanding of the problem
-----Original Message-----
From: Asselin, Ramy [mailto:ramy.asselin at hpe.com]
Sent: Wednesday, February 8, 2017 8:13 PM
To: Wang Shilong <wshilong at ddn.com>; openstack-infra at lists.openstack.org
Cc: Li Xi <lixi at ddn.com>
Subject: Re: [OpenStack-Infra] About setting up CI
Hi Wang,
The ip should be correct. It's possible that the ip in the log message is slightly different because it refers to a different instance floating ip?
If you have a nodepool image uploaded already, I suggest you try to manually launch a node using that image outside of nodepool. You should be able to ssh into it as Ubuntu user using the ssh private key provided when you created the image.
Ramy
-----Original Message-----
From: Wang Shilong [mailto:wshilong at ddn.com]
Sent: Wednesday, February 08, 2017 6:26 AM
To: openstack-infra at lists.openstack.org
Cc: Li Xi <lixi at ddn.com>
Subject: Re: [OpenStack-Infra] About setting up CI
Hello Guys,
I am following https://github.com/openstack-infra/puppet-openstackci/blob/master/doc/source/third_party_ci.rst
to setup CI.
I hit following problem from nodepool log.
2017-02-08 22:59:35,529 ERROR nodepool.utils: Exception while testing ssh access to 172.24.4.2:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/nodepool/nodeutils.py", line 55, in ssh_connect
client = SSHClient(ip, username, **connect_kwargs)
File "/usr/local/lib/python2.7/dist-packages/nodepool/sshclient.py", line 32, in __init__
allow_agent=allow_agent)
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 305, in connect
retry_on_signal(lambda: sock.connect(addr))
File "/usr/local/lib/python2.7/dist-packages/paramiko/util.py", line 270, in retry_on_signal
return function()
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 305, in <lambda>
retry_on_signal(lambda: sock.connect(addr))
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
error: [Errno 110] Connection timed out
2017-02-08 22:59:37,533 ERROR nodepool.NodeLauncher: Timeout launching node id: 4311 in provider: local_01 error: Timeout waiting for ssh access
After some checking, problem is openstack VM instance ip is not what openstack specified, for example for above case, indeed allocation ip of VM is 172.24.4.5.
So nodepool always failed because of this reasons, how can i slove this mismatch problems and proceed CI testing?
Thank you!
Shilong
________________________________________
From: Wang Shilong
Sent: Thursday, November 17, 2016 21:21
To: openstack-infra at lists.openstack.org
Cc: Li Xi
Subject: About setting up CI
Hello folks,
We are trying to setting up CI testing system for new Lustre cinder Drivers.
I try to do as following URL In centos7:
http://www.joinfu.com/2014/02/setting-up-an-external-openstack-testing-system/
But hit the following problem:
[root at vm7-1 ~]# ./install_master.sh
Using upstream Gerrit user: ddn_openstack Using Jenkins SSH key path: /root/data/jenkins_key
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Info: Loading facts
Error: Could not find template 'openstack_project/puppet.conf.erb' at /root/os-ext-testing/puppet/modules/os_ext_testing/manifests/base.pp:132 on node vm7-1
Error: Could not find template 'openstack_project/puppet.conf.erb' at /root/os-ext-testing/puppet/modules/os_ext_testing/manifests/base.pp:132 on node vm7-1
I guess this Doc might be out of date, could you please give me some hints how I can setup latest CI?
Thanks for your suggestion!
Shilong
_______________________________________________
OpenStack-Infra mailing list
OpenStack-Infra at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
_______________________________________________
OpenStack-Infra mailing list
OpenStack-Infra at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
More information about the OpenStack-Infra
mailing list