Open Stack

Hi Ramy,
    > This one you need to skip because you don’t have access to that server.
   OK, got it.

   Thanks a lot.

Xiexs


From: Asselin, Ramy [mailto:ramy.asselin at hp.com]
Sent: Wednesday, August 05, 2015 10:19 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [CI]How to set proxy for nodepool

HI Xiexs,

“Also, I’ve found some of the infra project-config elements don’t work in my environment and aren’t needed as they’re specific to infra. For those, simply comment out the portions that don’t work. I didn’t notice any negative side-effects.”
This one you need to skip because you don’t have access to that server.

In fact, here are the ones that I skip in my setup:

etc/nodepool/elements/nodepool-base/install.d/90-venv-swift-logs:# skipped
etc/nodepool/elements/nodepool-base/install.d/99-install-zuul:# Skipped
etc/nodepool/elements/nodepool-base/finalise.d/99-unbound:# Skipped
etc/nodepool/elements/cache-devstack/install.d/99-cache-testrepository-db:# Skipped

The ’99-unbound’ may work in your setup. If not, you need to disable it here too:
etc/nodepool/elements/puppet/bin/prepare-node:  enable_unbound => false,

Ramy

From: Xie, Xianshan [mailto:xiexs at cn.fujitsu.com]
Sent: Wednesday, August 05, 2015 3:40 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [CI]How to set proxy for nodepool

Hi Ramy,
Thanks for your patience.
I have tried your suggestion, but it did not work for me.
According to the log, this element has already ran in the chroot before the pip commands are executed.
So, in theory, the pip command  would run behind this proxy, but the connection errors are still raised.
It`s weird.

Then  I tried to hard code the proxy with --proxy option into the pip command, it works.
Anyway, this is a merely temporary solution for this issue until I figure it out.

But, after that, I got a new error:
-------------------------
nodepool.image.build.dpc: + sudo env 'PATH=/opt/git/subunit2sql-env/bin:/usr/lib64/ccache:/usr/lib/ccache:$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' /opt/git/subunit2sql-env/bin/python2 /opt/nodepool-scripts/prepare_tempest_testrepository.py /opt/git/openstack/tempest
nodepool.image.build.dpc: sudo: unable to resolve host fnst01
nodepool.image.build.dpc: No handlers could be found for logger "oslo_db.sqlalchemy.session"
nodepool.image.build.dpc: Traceback (most recent call last):
nodepool.image.build.dpc: File "/opt/nodepool-scripts/prepare_tempest_testrepository.py", line 50, in <module>
nodepool.image.build.dpc: main()
nodepool.image.build.dpc: File "/opt/nodepool-scripts/prepare_tempest_testrepository.py", line 39, in main
nodepool.image.build.dpc: session = api.get_session()
nodepool.image.build.dpc: File "/opt/git/subunit2sql-env/local/lib/python2.7/site-packages/subunit2sql/db/api.py", line 47, in get_session
nodepool.image.build.dpc: facade = _create_facade_lazily()
nodepool.image.build.dpc: File "/opt/git/subunit2sql-env/local/lib/python2.7/site-packages/subunit2sql/db/api.py", line 37, in _create_facade_lazily
nodepool.image.build.dpc: **dict(CONF.database.iteritems()))
nodepool.image.build.dpc: File "/opt/git/subunit2sql-env/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/session.py", line 822, in __init__
nodepool.image.build.dpc: **engine_kwargs)
nodepool.image.build.dpc: File "/opt/git/subunit2sql-env/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/session.py", line 417, in create_engine
nodepool.image.build.dpc: test_conn = _test_connection(engine, max_retries, retry_interval)
nodepool.image.build.dpc: File "/opt/git/subunit2sql-env/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/session.py", line 596, in _test_connection
nodepool.image.build.dpc: six.reraise(type(de_ref), de_ref)
nodepool.image.build.dpc: File "<string>", line 2, in reraise
nodepool.image.build.dpc: oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'logstash.openstack.org' ([Errno -5] No address associated with hostname)")
-------------------------
I think it is also a proxy problem about remote access to the subunit2sql database.
And it should work with simpleproxy I think.
But I couldn`t find how to use simpleproxy to forward data for the subunit2sql db.
Could you please give me more hints?

Thanks again.

Xiexs

From: Asselin, Ramy [mailto:ramy.asselin at hp.com]
Sent: Wednesday, August 05, 2015 6:00 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [CI]How to set proxy for nodepool

Hi Xiexs,

You might need to configure pip to use your proxy.

I added my own element here:

cache-devstack/install.d/98-setup-pip

Basically:

set -eux

mkdir -p /root/.pip/

cat <<EOF > /root/.pip/pip.conf
[global]
proxy = <your proxy>
EOF

cp -f /root/.pip/pip.conf /etc/

Ramy

From: Xie, Xianshan [mailto:xiexs at cn.fujitsu.com]
Sent: Tuesday, August 04, 2015 12:05 AM
To: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [CI]How to set proxy for nodepool

Hi Ramy,
Thanks for your help.

I have already confirmed proxy setting again, and it works fine(no matter whether the NODEPOOL_ variables are declared or not):

1)       not only in the host machine on which DIB run,

2)       but also in the first half part of DIB(before the DIB running chroot).

3)       furthmore, I ran the commands manually in the host env, they also works fine.

$ sudo -H virtualenv /usr/zuul-swift-logs-env

$ sudo -H /usr/zuul-swift-logs-env/bin/pip install python-magic argparse requests glob2

So, if I understood correctly, it seems obvious, that the proxy setting is missed when DIB goes into chroot env.
Thus, when DIB attempts to connect internet to download/install/update some materials to prepare the image within the chroot env, the error will be encountered.
In this case, DIB will run “pip install” in the chroot env zuul-swift-logs-env to install python-magic, argparse and so forth.

Actually, all NODEPOOL_ variables were already declared by the install_master.sh previously, and the proxy setting also derived from the host machine`s proxy setting.

Xiexs
From: Asselin, Ramy [mailto:ramy.asselin at hp.com]
Sent: Tuesday, August 04, 2015 12:54 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [CI]How to set proxy for nodepool

Hi Xiexs,

You are using disk image builder (DIB), so make sure your proxy is set correctly also without the NODEPOOL_ prefix.
Those NODEPOOL_ prefixed variables are what would be used when DIB is not used / when the script approach is used with nova/glance (nodepool image-update). I don’t recommend using that if you can use DIB.

Also, I’ve found some of the infra project-config elements don’t work in my environment and aren’t needed as they’re specific to infra. For those, simply comment out the portions that don’t work. I didn’t notice any negative side-effects.

Ramy

From: Xie, Xianshan [mailto:xiexs at cn.fujitsu.com]
Sent: Monday, August 03, 2015 9:37 PM
To: openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>
Subject: [openstack-dev] [CI]How to set proxy for nodepool

Hi All,
When I ran “nodepool image-build <image-name>” to create a new image, I got this error message:
…
INFO nodepool.image.build.dpc: + sudo -H virtualenv /usr/zuul-swift-logs-env
INFO nodepool.image.build.dpc: sudo: unable to resolve host fnst01
INFO nodepool.image.build.dpc: + sudo -H /usr/zuul-swift-logs-env/bin/pip install python-magic argparse requests glob2
INFO nodepool.image.build.dpc: sudo: unable to resolve host fnst01
INFO nodepool.image.build.dpc: Collecting python-magic
INFO nodepool.image.build.dpc: Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', gaierror(-5, 'No address associated with hostname'))': /simple/python-magic/
INFO nodepool.image.build.dpc: Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', gaierror(-5, 'No address associated with hostname'))': /simple/python-magic/
…

And I exported the bash env when the script elements/nodepool-base/install.d/90-venv-swift-logs ran,
it seems that the nodepool proxy and dns has already been set:

declare -x NODEPOOL_HTTPS_PROXY=http://xx.xx.xx.xx:8080/  <--valid proxy
declare -x NODEPOOL_HTTP_PROXY="http://xx.xx.xx.xx:8080/"
declare -x NODEPOOL_NO_PROXY="localhost,127.0.0.1"
declare -x NODEPOOL_SCRIPTDIR="/etc/nodepool/scripts"
declare -x NODEPOOL_STATIC_NAMESERVER="xx.xx.xx.xx"
declare -x PS1="(zuul-swift-logs-env)"
INFO nodepool.image.build.dpc: + cat /etc/resolv.conf
INFO nodepool.image.build.dpc: nameserver xx.xx.xx.xx       <--valid dns

And I have tried to add element “local-config” into nodepool.yaml, but it does not work.

So, my questions are:

1)      These bash env will be passed into the virtualenv zuul-swift-logs-env after chroot, is it right?

2)      If not, how to configure the proxy for nodepool?

Many thanks to any suggestion.

Xiexs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150806/09bfe0ef/attachment.html>