[Openstack-operators] [openstack-ansible] pip problems with openstack-ansible deployment
Danil Zhigalin (Europe)
Danil.Zhigalin at dimensiondata.com
Fri Feb 17 13:14:02 UTC 2017
So as I understood, autoindex should work for os-releases/14.0.7 but it is not working. There was an index.html file in that directory. I decided to delete it. After that autoindex started to do the job for that directory and I could see the list of all wheels.
Then I rerun the playbook and it installed everything from the repo.
I don’t know how that index.html got into os-releases/14.0.7, but it was the cause of the problem. I will check if it gets there after new deployment.
Thank you for your help!
Best regards,
Danil
From: Jesse Pretorius [mailto:Jesse.Pretorius at rackspace.co.uk]
Sent: 17 February 2017 12:04
To: openstack-operators at lists.openstack.org
Subject: Re: [Openstack-operators] [openstack-ansible] pip problems with openstack-ansible deployment
Hmm, that is odd.
OK, so to clarify some things:
There should be no /os-releases/<tag>/links – but there should be a /links – both of these contain symbolic links to the /pools directory. The /links folder should contain symlinks for *everything* in the /pools/ folder, whereas the /os-releases/<tag>/links folder should container *only* the links to wheels that are necessary for that tag. The /links folder is used for building wheels/venvs and the os-releases folder is used when doing pip installs. The folder structure is similar to an Ubuntu repo and is designed to reduce space usage over many releases (there is only ever one file for one wheel version, even if used over multiple releases).
Also, the index file should be an autoindex produced by the web server. If not, then something is wonky.
There is a known bug that was fixed recently and will be in the next tag release (this has been requested today and will likely release early next week): https://github.com/openstack/openstack-ansible-repo_build/commit/db2ae5f7cacc3d85c126f56eaef9864d1f2354f7<https://github.com/openstack/openstack-ansible-repo_build/commit/db2ae5f7cacc3d85c126f56eaef9864d1f2354f7>
This doesn’t seem to relate to your situation though.
With the confirmation that your indexes are working properly again, can you paste a debug ansible output for the pip install that’s failing?
Thanks,
Jesse
IRC: odyssey4me
From: "Danil Zhigalin (Europe)" <Danil.Zhigalin at dimensiondata.com<mailto:Danil.Zhigalin at dimensiondata.com>>
Date: Friday, February 17, 2017 at 10:26 AM
To: Jesse Pretorius <Jesse.Pretorius at rackspace.co.uk<mailto:Jesse.Pretorius at rackspace.co.uk>>, "openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>" <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Subject: RE: [Openstack-operators] [openstack-ansible] pip problems with openstack-ansible deployment
Jus to test this I changed index.html that is returned from http://172.21.51.152:8181/os-releases/14.0.7/<http://172.21.51.152:8181/os-releases/14.0.7/> in the following way to include absolute paths there.
curl http://172.21.51.152:8181/os-releases/14.0.7/<http://172.21.51.152:8181/os-releases/14.0.7/>
<html>
<head><title>Index of /</title></head>
<body bgcolor="white">
<h1>Index of /</h1><hr><pre><a href="../">../</a>
<a href="/links/">/links/</a> 16-Feb-2017 11:03 -
<a href="/openstackgit/">/openstackgit/</a> 16-Feb-2017 10:20 -
<a href="/os-releases/">/os-releases/</a> 16-Feb-2017 10:17 -
<a href="/pkg-cache/">/pkg-cache/</a> 16-Feb-2017 10:18 -
<a href="/pools/">/pools/</a> 16-Feb-2017 10:28 -
<a href="/venvs/">/venvs/</a> 16-Feb-2017 10:28 -
</pre><hr></body>
</html>
Now links from this index.html work but pip still can’t locate the wheel there even though it is on the filesystem in this directory:
root at control1-repo-container-dad60ff0:~# ls /var/www/repo/os-releases/14.0.7 | grep mysql_
mysql_python-1.2.5-cp27-cp27mu-linux_x86_64.whl
Best regards,
Danil
From: Danil Zhigalin (Europe)
Sent: 17 February 2017 11:04
To: 'Jesse Pretorius' <Jesse.Pretorius at rackspace.co.uk<mailto:Jesse.Pretorius at rackspace.co.uk>>; openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>
Subject: RE: [Openstack-operators] [openstack-ansible] pip problems with openstack-ansible deployment
Hi Jesse,
Thanks for your reply. To keep things straight during my test deployment I took only one control node. So there is only one repo container behind the load balancer. What I don’t really understand, is why this index.html is returned when in both cases, either if I address
http://172.21.51.152:8181/<http://172.21.51.152:8181/>
curl http://172.21.51.152:8181/<http://172.21.51.152:8181/>
<html>
<head><title>Index of /</title></head>
<body bgcolor="white">
<h1>Index of /</h1><hr><pre><a href="../">../</a>
<a href="links/">links/</a> 16-Feb-2017 13:27 -
<a href="openstackgit/">openstackgit/</a> 16-Feb-2017 10:20 -
<a href="os-releases/">os-releases/</a> 16-Feb-2017 10:17 -
<a href="pkg-cache/">pkg-cache/</a> 17-Feb-2017 05:46 -
<a href="pools/">pools/</a> 16-Feb-2017 10:28 -
<a href="venvs/">venvs/</a> 16-Feb-2017 10:28 -
</pre><hr></body>
</html>
or
http://172.21.51.152:8181/os-releases/14.0.7/<http://172.21.51.152:8181/os-releases/14.0.7/>
curl http://172.21.51.152:8181/os-releases/14.0.7/<http://172.21.51.152:8181/os-releases/14.0.7/>
<html>
<head><title>Index of /</title></head>
<body bgcolor="white">
<h1>Index of /</h1><hr><pre><a href="../">../</a>
<a href="links/">links/</a> 16-Feb-2017 11:03 -
<a href="openstackgit/">openstackgit/</a> 16-Feb-2017 10:20 -
<a href="os-releases/">os-releases/</a> 16-Feb-2017 10:17 -
<a href="pkg-cache/">pkg-cache/</a> 16-Feb-2017 10:18 -
<a href="pools/">pools/</a> 16-Feb-2017 10:28 -
<a href="venvs/">venvs/</a> 16-Feb-2017 10:28 -
</pre><hr></body>
</html>
In fact, none of the links are working in the second case, since they are considered relative to the /os-releases/14.0.7/:
curl http://172.21.51.152:8181/os-releases/14.0.7/links/<http://172.21.51.152:8181/os-releases/14.0.7/links/>
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.4.6 (Ubuntu)</center>
</body>
</html>
Although in the ~/.pip/pip.conf I see the following:
[global]
no-index = true
pre = True
timeout = 120
trusted-host =
172.21.51.152
[install]
upgrade = True
find-links =
http://172.21.51.152:8181/os-releases/14.0.7/<http://172.21.51.152:8181/os-releases/14.0.7/>
From: Jesse Pretorius [mailto:Jesse.Pretorius at rackspace.co.uk]
Sent: 17 February 2017 10:43
To: openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>
Subject: Re: [Openstack-operators] [openstack-ansible] pip problems with openstack-ansible deployment
Hi Danil,
It sounds like it might be a problem with the repo containers not being in sync somehow. Can you verify that all the repo containers have the same content? The load balancer may be directing you to a container that doesn’t have that wheel for some reason.
HTH,
Jesse
IRC: odyssey4me
From: "Danil Zhigalin (Europe)" <Danil.Zhigalin at dimensiondata.com<mailto:Danil.Zhigalin at dimensiondata.com>>
Date: Friday, February 17, 2017 at 9:11 AM
To: Kenneth Tan <cjtan at sardinasystems.com<mailto:cjtan at sardinasystems.com>>, "openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>" <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Subject: Re: [Openstack-operators] [openstack-ansible] pip problems with openstack-ansible deployment
Hi Kenneth,
Exactly, it is my goal. Exploring different OpenStack deployment tools. Found Openstack-Ansible project (https://github.com/openstack/openstack-ansible<https://github.com/openstack/openstack-ansible>) and like what it delivers (containerized or bare metal deployment). Unfortunately, can’t figure out how to advance further due to that problem.
Danil Zhigalin
Technical Consultant
Dimension Data Germany
Tel: +49 211 1717 1260
Mob: +49 174 151 8457
Danil.Zhigalin at dimensiondata.com<mailto:Danil.Zhigalin at dimensiondata.com>
Dimension Data Germany AG & Co. KG, Derendorfer Allee 26, 40476, Düsseldorf, North Rhine-Westphalia, Germany.
For more information, please go to www.dimensiondata.com<http://www.dimensiondata.com/>
[tp://de.blog.dimensiondata.com/]<http://de.blog.dimensiondata.com/> [tps://www.facebook.com/Dimension-Data-Germany-921346461234757/] <https://www.facebook.com/Dimension-Data-Germany-921346461234757/> [tps://www.linkedin.com/company/dimension-data] <https://www.linkedin.com/company/dimension-data> [tps://twitter.com/DimensionDataDe] <https://twitter.com/DimensionDataDe> [cid:image005.png at 01D2891C.D9E57970] <https://www.xing.com/companies/dimensiondata>
[cid:image006.jpg at 01D2891C.D9E57970]<http://>
Dimension Data Germany AG & Co.KG, Horexstraße 7, 61352 Bad Homburg
Sitz: Bad Homburg, Amtsgericht Bad Homburg, HRA 3207
Pers. Haftende Ges : Dimension Data Verwaltungs AG, Sitz Bad Homburg.
Amtsgericht Bad Homburg, HRB 6172
Vorstand: Roberto Del Corno
Vors. des Aufsichtsrats: Andrew Coulsen
From: Kenneth Tan [mailto:cjtan at sardinasystems.com]
Sent: 17 February 2017 09:22
To: Danil Zhigalin (Europe) <Danil.Zhigalin at dimensiondata.com<mailto:Danil.Zhigalin at dimensiondata.com>>
Subject: Re: [Openstack-operators] pip problems with openstack-ansible deployment
Hi Danil,
I am curious, what is your end objective? Is the objective to deploy an OpenStack cloud using an Ansible-based tool?
Best wishes,
Ken
-----------------
Kenneth Tan
Sardina Systems
E: cjtan at sardinasystems.com<mailto:cjtan at sardinasystems.com>
M: +447989417838
T: +442070994428
W: www.sardinasystems.com<http://www.sardinasystems.com>
On 17 Feb 2017, at 08:15, Danil Zhigalin (Europe) <Danil.Zhigalin at dimensiondata.com<mailto:Danil.Zhigalin at dimensiondata.com>> wrote:
Hello everyone,
Context:
openstact-ansible: stable/newton
OS: ubuntu 16.04 LTS
I am having trouble completing my deployment due to pip errors.
I have a 2 node setup and one separate deployment node. One of the nodes I am using to host all controller, network and storage functions and another as a compute. Repo container with the server is also hosted on the controller node. I already ran into similar problems as Achi Hamza who already reported pip issue on the Thu Nov 17 08:34:14 UTC 2016 in this mailing list.
This is how my openstack_user_config.yml file looks like (as in Hamza's case internal and external addresses are the same):
global_overrides:
internal_lb_vip_address: 172.21.51.152
external_lb_vip_address: 172.21.51.152 <...>
The recommendation that he got from another users were to set:
openstack_service_publicuri_proto: http
openstack_external_ssl: false
haproxy_ssl: false
in /etc/openstack_deploy/user_vriables.yml
These recommendations helped in my case as well and I was able to advance further until I faced another pip issues in the same playbook.
My current problem is that neither of containers can install pip packages from the repository.
TASK [galera_client : Install pip packages] ************************************
FAILED - RETRYING: TASK: galera_client : Install pip packages (5 retries left).
FAILED - RETRYING: TASK: galera_client : Install pip packages (4 retries left).
FAILED - RETRYING: TASK: galera_client : Install pip packages (3 retries left).
FAILED - RETRYING: TASK: galera_client : Install pip packages (2 retries left).
FAILED - RETRYING: TASK: galera_client : Install pip packages (1 retries left).
fatal: [control1_galera_container-434df170]: FAILED! => {"changed": false, "cmd": "/usr/local/bin/pip install -U --constraint http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt<http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt> MySQL-python", "failed": true, "msg": "stdout: Collecting mysql_python==1.2.5 (from -c http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt<http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt> (line 81))\n\n:stderr: Could not find a version that satisfies the requirement mysql_python==1.2.5 (from -c http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt<http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt> (line 81)) (from versions: )\nNo matching distribution found for mysql_python==1.2.5 (from -c http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt<http://172.21.51.152:8181/os-releases/14.0.7/requirements_absolute_requirements.txt> (line 81))\n"}
I already checked everything related to the HAproxy and tcpdumped on the repo side to see what requests are coming when pip install is called.
I found that there was a HTTP GET to the URL http://172.21.51.152:8181/os-releases/14.0.7/<http://172.21.51.152:8181/os-releases/14.0.7/>
I saw that it was forwarded by the proxy to the repo server and that repo server returned index.html from /var/www/repo/os-releases/14.0.7/
ls /var/www/repo/os-releases/14.0.7/ | grep index index.html
index.html.1
index.html.2
I also checked that MySQL-python is in the repo:
root at control1-repo-container-dad60ff0:~# ls /var/www/repo/os-releases/14.0.7/ | grep mysql_python mysql_python-1.2.5-cp27-cp27mu-linux_x86_64.whl
But for some reason pip can't figure out it is there.
I very much appreciate your help in solving this issue.
Best regards,
Danil
Danil Zhigalin
Technical Consultant
Tel: +49 211 1717 1260
Mob: +49 174 151 8457
danil.zhigalin at dimensiondata.com<mailto:danil.zhigalin at dimensiondata.com>
Derendorfer Allee 26, Düsseldorf, North Rhine-Westphalia, 40476, Germany.
For more information, please go to www.dimensiondata.com<http://www.dimensiondata.com/>
Dimension Data Germany AG & Co.KG, Horexstraße 7, 61352 Bad Homburg
Sitz: Bad Homburg, Amtsgericht Bad Homburg, HRA 3207
Pers. Haftende Ges : Dimension Data Verwaltungs AG, Sitz Bad Homburg.
Amtsgericht Bad Homburg, HRB 6172
Vorstand: Roberto Del Corno
Vors. des Aufsichtsrats: Andrew Coulsen.
This email and all contents are subject to the following disclaimer:
"http://www.dimensiondata.com/emaildisclaimer"<http://www.dimensiondata.com/Global/Policies/Pages/Email-Disclaimer.aspx>
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators>
itevomcid
________________________________
Rackspace Limited is a company registered in England & Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy<http://www.rackspace.co.uk/legal/privacy-policy> - This e-mail message may contain confidential or privileged information intended for the recipient. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at abuse at rackspace.com<mailto:abuse at rackspace.com> and delete the original message. Your cooperation is appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 1663 bytes
Desc: image001.jpg
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0005.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 1435 bytes
Desc: image002.jpg
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0006.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 1491 bytes
Desc: image003.jpg
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0007.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 1463 bytes
Desc: image004.jpg
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0008.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 2134 bytes
Desc: image005.png
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 81062 bytes
Desc: image006.jpg
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170217/bc5b6948/attachment-0009.jpg>
More information about the OpenStack-operators
mailing list
