[Openstack-operators] [openstack-dev] [openstack-ansible] pip issues

Achi Hamza h16mara at gmail.com
Thu Nov 17 11:22:38 UTC 2016


Thank you for the remainder regarding the mailing list, i just forgot about
that.

Yes, i can reach them but without data form curl:

*root at maas:/opt/openstack-ansible/playbooks# ansible repo_all -m shell -a
"curl
http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt
<http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt>"*
Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e
@/etc/openstack_deploy/user_variables.yml "
node01_repo_container-82b4e1f6 | FAILED | rc=52 >>
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
 Current
                                 Dload  Upload   Total   Spent    Left
 Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
  0
curl: (52) Empty reply from server

*root at maas:/opt/openstack-ansible/playbooks# ansible hosts -m shell -a
"curl
http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt
<http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt>"*
Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e
@/etc/openstack_deploy/user_variables.yml "
node01 | FAILED | rc=52 >>
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
 Current
                                 Dload  Upload   Total   Spent    Left
 Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
  0
curl: (52) Empty reply from server

*root at node01:~# curl
http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt
<http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt>*
curl: (52) Empty reply from server

I have set these roles with my iptables earlier (this is just for the nodes
to get out through the deployment node), can this have an impact ? :

iptables -A FORWARD -o enp4s0 -i enp5s0 -s 172.16.1.1/24 -m conntrack
--ctstate NEW -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -F POSTROUTING
iptables -t nat -A POSTROUTING -o enp4s0 -j MASQUERADE

Thank you


On 17 November 2016 at 12:06, Jesse Pretorius <
Jesse.Pretorius at rackspace.co.uk> wrote:

> Please continue to make use of the list instead of going off list. The
> list archives are a reference point for others who may find themselves in
> the same situation.
>
>
>
> Right, so the service is up. The load balancer is up, so we’re down to
> networking issues or perhaps something a little more obscure.
>
>
>
> Can you confirm whether you’re able to reach the repo contents from each
> of the following points: repo container, repo container host, deployment
> host
>
>
>
> ie
>
>
>
> ansible repo_all -m shell -a "curl http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt"
>
> ansible hosts -m shell -a "curl http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt"
>
> curl http://172.16.1.2:8181/os-releases/14.0.1/requirements_
> constraints.txt
>
>
>
> J
>
>
>
> *From: *Achi Hamza <h16mara at gmail.com>
> *Date: *Thursday, November 17, 2016 at 10:59 AM
> *To: *Jesse Pretorius <Jesse.Pretorius at rackspace.co.uk>
>
> *Subject: *Re: [Openstack-operators] [openstack-dev] [openstack-ansible]
> pip issues
>
>
>
> Thank you Jesse.
>
>
>
> here are the output of the 2 commands:
>
>
>
> nginx server is UP:
>
> r*oot at maas:/opt/openstack-ansible/playbooks# ansible repo_all -m shell -a
> "service nginx status"*
>
> *Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e
> @/etc/openstack_deploy/user_variables.yml "*
>
> node01_repo_container-82b4e1f6 | SUCCESS | rc=0 >>
>
> ● nginx.service - A high performance web server and a reverse proxy server
>
>    Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor
> preset: enabled)
>
>    Active: active (running) since Wed 2016-11-16 13:12:57 UTC; 21h ago
>
>  Main PID: 2729 (nginx)
>
>     Tasks: 3
>
>    Memory: 956.0K
>
>       CPU: 5.832s
>
>    CGroup: /system.slice/nginx.service
>
>            ├─2729 nginx: master process /usr/sbin/nginx -g daemon on;
> master_process on
>
>            ├─2730 nginx: worker process
>
>            └─2731 nginx: worker process
>
>
>
> Nov 16 13:12:57 node01-repo-container-82b4e1f6 systemd[1]: Starting A high
> performance web server and a reverse proxy server...
>
> Nov 16 13:12:57 node01-repo-container-82b4e1f6 systemd[1]: Started A high
> performance web server and a reverse proxy server.
>
>
>
> the repo-all* is also UP:
>
> *root at node01:~# hatop -s /var/run/haproxy.stat*
>
> >>> rabbitmq_mgmt-back
>
> ..ner-2148a5b8        1 DOWN       L4CON             1      0        0
>    0         0         0         0         0
>
> BACKEND               0 DOWN                         0      0        0
>    0         0         0       410         0
>
>
>
> >>> repo_all-front-1
>
> FRONTEND              0 OPEN                         0      0        0
>    0         0         1      4096         0
>
>
>
> >>> repo_all-back
>
> ..ner-82b4e1f6        1 UP         L7OK              1      0        0
>    0         0         0         0         0
>
> BACKEND               1 UP                           1      0        0
>    0         0         0       410         0
>
>
>
> >>> repo_cache-front-1
>
> FRONTEND              0 OPEN                         0      0        0
>    0         0         1      4096         0
>
>
>
> >>> repo_cache-back
>
> ..ner-82b4e1f6        1 UP         L7OK              1      0        0
>    0         0         0         0         0
>
> BACKEND               1 UP                           1      0        0
>    0         0         0       410         0
>
>
>
> >>> repo_git-front-1
>
> FRONTEND              0 OPEN                         0      0        0
>    0         0         0      4096         0
>
>
>
> >>> repo_git-back
>
> ..ner-82b4e1f6        1 UP         L4OK              1      0        0
>    0         0         0         0         0
>
> BACKEND               1 UP                           1      0        0
>    0         0         0       410         0
>
>
>
>  1-STATUS  2-TRAFFIC  3-HTTP  4-ERRORS  5-CLI
>                                     [#30/#0]
>
>
>
> What else please ?
>
>
>
> On 17 November 2016 at 11:47, Jesse Pretorius <
> Jesse.Pretorius at rackspace.co.uk> wrote:
>
> OK, so it’s clearly there in terms of data.
>
>
>
> Can you now verify whether the web server is up on the repo server?
>
>
>
> ansible repo_all -m shell -a "service nginx status"
>
>
>
> Also check haproxy by accessing the haproxy host and running:
>
>
>
> hatop –s /var/run/haproxy.stat
>
> Look specifically for the repo_all-* entries and check whether they’re
> up/down.
>
> There should be at least one front end and one back end.
>
>
>
> Thanks,
>
>
>
> Jesse
>
>
>
> *From: *Achi Hamza <h16mara at gmail.com>
> *Date: *Thursday, November 17, 2016 at 10:39 AM
> *To: *Jesse Pretorius <Jesse.Pretorius at rackspace.co.uk>
> *Cc: *"OpenStack-operators at lists.openstack.org" <
> OpenStack-operators at lists.openstack.org>
>
>
> *Subject: *Re: [Openstack-operators] [openstack-dev] [openstack-ansible]
> pip issues
>
>
>
> Hi Jesse,
>
>
>
> Yes, i can:
>
>
>
> root at maas:/opt/openstack-ansible/playbooks# ansible repo_all -m shell -a
> "find /var/www/repo/os-releases -type f"
>
> Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e
> @/etc/openstack_deploy/user_variables.yml "
>
> node01_repo_container-82b4e1f6 | SUCCESS | rc=0 >>
>
> /var/www/repo/os-releases/14.0.1/requirements.txt
>
> /var/www/repo/os-releases/14.0.1/requirements_absolute_requirements.txt
>
> /var/www/repo/os-releases/14.0.1/MANIFEST.in
>
> /var/www/repo/os-releases/14.0.1/index.html
>
> /var/www/repo/os-releases/14.0.1/requirements_constraints.txt
>
>
>
>
>
> On 17 November 2016 at 11:35, Jesse Pretorius <
> Jesse.Pretorius at rackspace.co.uk> wrote:
>
> Hi Achi,
>
>
>
> Can you list the files in /var/www/repo/os-release?
>
>
>
> ansible repo_all -m shell -a "find /var/www/repo/os-releases -type f"
>
>
>
> Thanks,
>
>
>
> Jesse
>
> IRC: odyssey4me
>
>
>
> *From: *Achi Hamza <h16mara at gmail.com>
> *Date: *Thursday, November 17, 2016 at 10:01 AM
> *To: *"OpenStack-operators at lists.openstack.org" <
> OpenStack-operators at lists.openstack.org>, "openstack-dev at lists.
> openstack.org" <openstack-dev at lists.openstack.org>
> *Subject: *Re: [Openstack-operators] [openstack-dev] [openstack-ansible]
> pip issues
>
>
>
> Hi Jesse and Jean-Philippe,
>
>
>
> Thank you both for your responses.
>
>
>
> @Jesse
>
> I am just having one node for the infra and one repo, for now:
>
>
>
>  root at maas:/opt/openstack-ansible/playbooks# ansible repo_all -m shell -a
> "du -sh /var/www/repo/*"
>
> Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e
> @/etc/openstack_deploy/user_variables.yml "
>
> node01_repo_container-82b4e1f6 | SUCCESS | rc=0 >>
>
> 900K
>
> /var/www/repo/links
>
> 656M
>
> /var/www/repo/openstackgit
>
> 964K
>
> /var/www/repo/os-releases
>
> 4.0K
>
> /var/www/repo/pkg-cache
>
> 105M
>
> /var/www/repo/pools
>
> 273M
>
> /var/www/repo/venvs
>
>
>
> @Jean-Philippe
>
> Yes, i am using haproxy. I cannot wget the http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt file neither form the bare
> metal node nor from within the container:
>
>
>
> root at node01-galera-container-368f269a:~# wget -c
> http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt
>
> --2016-11-17 09:53:19--  http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt
>
> Connecting to 172.16.1.2:8181... connected.
>
> HTTP request sent, awaiting response... No data received.
>
> Retrying.
>
> --2016-11-17 09:53:20--  (try: 2)  http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt
>
> Connecting to 172.16.1.2:8181... connected.
>
> HTTP request sent, awaiting response... No data received.
>
> Retrying.
>
> --2016-11-17 09:53:22--  (try: 3)  http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt
>
> Connecting to 172.16.1.2:8181... connected.
>
> HTTP request sent, awaiting response... No data received.
>
> Retrying.
>
>
>
> I attached my haproxy.conf file, can you please check it ? If the problem
> persists i will pop into the irc channel.
>
>
>
> Thank you,
>
> Hamza
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 17 November 2016 at 10:10, Jean-Philippe Evrard <Jean-Philippe.Evrard@
> rackspace.co.uk> wrote:
>
> Hello,
>
>
>
> Is this a greenfield newton?
>
> Could you wget your http://172.16.1.2:8181/os-
> releases/14.0.1/requirements_constraints.txt  file reliably?
>
> Are you using haproxy?  Are all the backends OK there?
>
>
>
> Don’t hesitate to come by our irc channel, it’s probably easier to have a
> conversation there.
>
>
>
> Best regards,
>
> Jean-Philippe
>
>
>
> *From: *Achi Hamza <h16mara at gmail.com>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev at lists.openstack.org>
> *Date: *Thursday, 17 November 2016 at 08:56
> *To: *"openstack-dev at lists.openstack.org" <openstack-dev at lists.
> openstack.org>
> *Subject: *[openstack-dev] [openstack-ansible] pip issues
>
>
>
> Hi all,
>
> Context:
> openstact-ansible newton, tag: 14.0.1
> OS: ubuntu 16.04 LTS
>
> the pip command is unable to download packages form the repo container. I
> can reach the repo container from within galera container through the port
> 8181:
>
>
>
> *root at node01:~# lxc-attach -n node01_galera_container-368f269a
> root at node01-galera-container-368f269a:~# nc -nzv 172.16.1.2 8181 Connection
> to 172.16.1.2 8181 port [tcp/*] succeeded!*
>
> However, when i run galera-install.yml playbook it throws the following
> error (but the root cause is pip_install role)  :
> *root at maas:/opt/openstack-ansible/playbooks# openstack-ansible
> galera-install.yml*
> ……….........................................................
> .....................
>
>
>
>
>
> *TASK [pip_install : Install pip packages (fall back mode)]
> ********************* FAILED - RETRYING: TASK: pip_install : Install pip
> packages (fall back mode) (5 retries left). FAILED - RETRYING: TASK:
> pip_install : Install pip packages (fall back mode) (4 retries left).
> FAILED - RETRYING: TASK: pip_install : Install pip packages (fall back
> mode) (3 retries left). FAILED - RETRYING: TASK: pip_install : Install pip
> packages (fall back mode) (2 retries left). FAILED - RETRYING: TASK:
> pip_install : Install pip packages (fall back mode) (1 retries left).fatal:
> [node01_galera_container-368f269a]: FAILED! => {"changed": false, "cmd":
> "/usr/local/bin/pip install -U --isolated
> --constraint http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt
> <http://172.16.1.2:8181/os-releases/14.0.1/requirements_constraints.txt>  ndg-httpsclient
> requests", "failed": true, "msg": "\n:stderr: Retrying (Retry(total=4,
> connect=None, read=None, redirect=None)) after connection broken by
> 'ProtocolError('Connection aborted.', BadStatusLine(\"''\",))':
> /os-releases/14.0.1/requirements_constraints.txt\nRetrying (Retry(total=3,
> connect=None, read=None, redirect=None)) after connection broken by
> 'ProtocolError('Connection aborted.', BadStatusLine(\"''\",))':
> /os-releases/14.0.1/requirements_constraints.txt\nRetrying (Retry(total=2,
> connect=None, read=None, redirect=None)) after connection broken by
> 'ProtocolError('Connection aborted.', BadStatusLine(\"''\",))':
> /os-releases/14.0.1/requirements_constraints.txt\nRetrying (Retry(total=1,
> connect=None, read=None, redirect=None)) after connection broken by
> 'ProtocolError('Connection aborted.', BadStatusLine(\"''\",))':
> /os-releases/14.0.1/requirements_constraints.txt\nRetrying (Retry(total=0,
> connect=None, read=None, redirect=None)) after connection broken by
> 'ProtocolError('Connection aborted.', BadStatusLine(\"''\",))':
> /os-releases/14.0.1/requirements_constraints.txt\nException:\nTraceback
> (most recent call last):\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/basecommand.py\", line 215, in
> main\n    status = self.run(options, args)\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/commands/install.py\", line
> 287, in run\n    wheel_cache\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/basecommand.py\", line 269, in
> populate_requirement_set\n    session=session, wheel_cache=wheel_cache):\n
>  File \"/usr/local/lib/python2.7/dist-packages/pip/req/req_file.py\", line
> 84, in parse_requirements\n    filename, comes_from=comes_from,
> session=session\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/download.py\", line 409, in
> get_file_content\n    resp = session.get(url)\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/sessions.py\",
> line 487, in get\n    return self.request('GET', url, **kwargs)\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/download.py\", line 378, in
> request\n    return super(PipSession, self).request(method, url, *args,
> **kwargs)\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/sessions.py\",
> line 475, in request\n    resp = self.send(prep, **send_kwargs)\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/sessions.py\",
> line 585, in send\n    r = adapter.send(request, **kwargs)\n  File
> \"/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/adapters.py\",
> line 467, in send\n    raise ConnectionError(e,
> request=request)\nConnectionError: HTTPConnectionPool(host='172.16.1.2',
> port=8181): Max retries exceeded with url:
> /os-releases/14.0.1/requirements_constraints.txt (Caused by
> ProtocolError('Connection aborted.', BadStatusLine(\"''\",)))\nYou are
> using pip version 8.1.2, however version 9.0.1 is available.\nYou should
> consider upgrading via the 'pip install --upgrade pip' command.\n"}*
>
>
>
> Please advice on what i am doing wrong. I am stuck :)
>
> Thank you,
> Hamza
>
>
> ------------------------------
>
> Rackspace Limited is a company registered in England & Wales (company
> registered number 03897010) whose registered office is at 5 Millington
> Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy
> can be viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail
> message may contain confidential or privileged information intended for the
> recipient. Any dissemination, distribution or copying of the enclosed
> material is prohibited. If you receive this transmission in error, please
> notify us immediately by e-mail at abuse at rackspace.com and delete the
> original message. Your cooperation is appreciated.
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161117/189b5778/attachment-0001.html>


More information about the OpenStack-operators mailing list