[novnc-console] Cannot connect to console
Hi all, This is my first post to this list so excuse me if I have not submitted correctly. I have installed openstack Victoria manually as a multi node setup. A controller & 3 computes. Everything works fine and the way it's expected. I have secured horizon with letsencrypt certs (for now) and again all is fine. When I did a test deploy I also used those certs to load the novnc console securely and it worked. My problem with my new deploy is that the console will not load no matter what I try. I get the following error when I enable debug mode in nova. 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy Traceback (most recent call last):2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy File "/usr/lib/python3/dist-packages/websockify/websockifyserver.py", line 691, in top_new_client2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy client = self.do_handshake(startsock, address)2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy File "/usr/lib/python3/dist-packages/websockify/websockifyserver.py", line 578, in do_handshake2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy context.load_cert_chain(certfile=self.cert, keyfile=self.key, password=self.key_password)2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy PermissionError: [Errno 13] Permission denied If I don't have debug enabled I just get the permission denied error. I have switched to the nova user and confirmed I can access the certs directory and read the certs. All my nova services are running fine as well. My controller conf is the following:[default]ssl_only=true cert=/etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/fullchain.pemkey=/etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/privkey.pem [vnc]enabled = trueserver_listen = 0.0.0.0server_proxyclient_address = $my_ipnovncproxy_base_url = https://<MY_CONTROLLER_FQDN>:6080/vnc_auto.html My compute config is the following:[vnc]enabled = trueserver_listen = 0.0.0.0server_proxyclient_address = $my_ipnovncproxy_base_url = https://<MY_CONTROLLER_FQDN>:6080/vnc_auto.html If anyone could help that would be really appreciated or any advice to further troubleshoot!! I cannot see anything else in any logs but I might not be looking in the right place. Thank you in advance. Derek
Hi Derek, What's the permissions of the letsencrypt cert files and the user that Nova is running on? sudo -u nova stat /etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/fullchain.pem Will probably fail, so you might wanna fix that! M On Fri, Jun 4, 2021 at 11:23 AM Derek O keeffe <derekokeeffe85@yahoo.ie> wrote:
Hi all,
This is my first post to this list so excuse me if I have not submitted correctly.
I have installed openstack Victoria manually as a multi node setup. A controller & 3 computes. Everything works fine and the way it's expected. I have secured horizon with letsencrypt certs (for now) and again all is fine. When I did a test deploy I also used those certs to load the novnc console securely and it worked.
My problem with my new deploy is that the console will not load no matter what I try. I get the following error when I enable debug mode in nova.
2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy Traceback (most recent call last): 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy File "/usr/lib/python3/dist-packages/websockify/websockifyserver.py", line 691, in top_new_client 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy client = self.do_handshake(startsock, address) 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy File "/usr/lib/python3/dist-packages/websockify/websockifyserver.py", line 578, in do_handshake 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy context.load_cert_chain(certfile=self.cert, keyfile=self.key, password=self.key_password) 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy PermissionError: [Errno 13] Permission denied
If I don't have debug enabled I just get the permission denied error. I have switched to the nova user and confirmed I can access the certs directory and read the certs. All my nova services are running fine as well.
My controller conf is the following: [default] ssl_only=true cert=/etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/fullchain.pem key=/etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/privkey.pem
[vnc] enabled = true server_listen = 0.0.0.0 server_proxyclient_address = $my_ip novncproxy_base_url = https://<MY_CONTROLLER_FQDN>:6080/vnc_auto.html
My compute config is the following: [vnc] enabled = true server_listen = 0.0.0.0 server_proxyclient_address = $my_ip novncproxy_base_url = https://<MY_CONTROLLER_FQDN>:6080/vnc_auto.html
If anyone could help that would be really appreciated or any advice to further troubleshoot!! I cannot see anything else in any logs but I might not be looking in the right place. Thank you in advance.
Derek
-- Mohammed Naser VEXXHOST, Inc.
Hi Mohammad, Thanks you for the reply. Below is the output of the command you sent: sudo -u nova stat /etc/letsencrypt/live/<MY_CONTROLLER_FQDN>/fullchain.pem File: /etc/letsencrypt/live/<MY_CONTROLLER_FQDN>/fullchain.pem Size: 5616 Blocks: 16 IO Block: 4096 regular fileDevice: 802h/2050d Inode: 7340138 Links: 1Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)Access: 2021-06-04 15:47:48.544545426 +0100Modify: 2021-06-03 11:50:26.410071017 +0100Change: 2021-06-03 11:52:39.870554481 +0100 Birth: - The permissions on the live directory are: ls -al /etc/letsencrypt/live/total 16drwx--x--x 3 root root 4096 Jun 3 11:53 .drwxr-xr-x 9 root root 4096 Jun 3 11:50 ..-rw-r--r-- 1 root root 740 Jun 3 11:50 READMEdrwxr-xr-x 2 root root 4096 Jun 3 11:50 <MY_CONTROLLER_FQDN> I changed the owner and group to nova as a test to see if that was the issue but it still didn't work. The first error I had was as you say a permissions issue on the live directory and as nova (su nova -s /bin/bash) I couldn't access that directory so I changed the permissions and tested it as the nova user (cd /etc/letsencrypt/live & cat fullchain.pem) and I could read the files in there. I then had the error I sent in the original email. The funny thing is I had a test deploy and it all worked fine but when I redeployed it on new machines with the same OS (ubuntu 20.04) it won't work for me. Regards,Derek On Friday 4 June 2021, 19:59:31 IST, Mohammed Naser <mnaser@vexxhost.com> wrote: Hi Derek, What's the permissions of the letsencrypt cert files and the user that Nova is running on? sudo -u nova stat /etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/fullchain.pem Will probably fail, so you might wanna fix that! M On Fri, Jun 4, 2021 at 11:23 AM Derek O keeffe <derekokeeffe85@yahoo.ie> wrote:
Hi all,
This is my first post to this list so excuse me if I have not submitted correctly.
I have installed openstack Victoria manually as a multi node setup. A controller & 3 computes. Everything works fine and the way it's expected. I have secured horizon with letsencrypt certs (for now) and again all is fine. When I did a test deploy I also used those certs to load the novnc console securely and it worked.
My problem with my new deploy is that the console will not load no matter what I try. I get the following error when I enable debug mode in nova.
2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy Traceback (most recent call last): 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy File "/usr/lib/python3/dist-packages/websockify/websockifyserver.py", line 691, in top_new_client 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy client = self.do_handshake(startsock, address) 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy File "/usr/lib/python3/dist-packages/websockify/websockifyserver.py", line 578, in do_handshake 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy context.load_cert_chain(certfile=self.cert, keyfile=self.key, password=self.key_password) 2021-06-04 15:54:11.004 356545 ERROR nova.console.websocketproxy PermissionError: [Errno 13] Permission denied
If I don't have debug enabled I just get the permission denied error. I have switched to the nova user and confirmed I can access the certs directory and read the certs. All my nova services are running fine as well.
My controller conf is the following: [default] ssl_only=true cert=/etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/fullchain.pem key=/etc/letsencrypt/live/ <MY_CONTROLLER_FQDN>/privkey.pem
[vnc] enabled = true server_listen = 0.0.0.0 server_proxyclient_address = $my_ip novncproxy_base_url = https://<MY_CONTROLLER_FQDN>:6080/vnc_auto.html
My compute config is the following: [vnc] enabled = true server_listen = 0.0.0.0 server_proxyclient_address = $my_ip novncproxy_base_url = https://<MY_CONTROLLER_FQDN>:6080/vnc_auto.html
If anyone could help that would be really appreciated or any advice to further troubleshoot!! I cannot see anything else in any logs but I might not be looking in the right place. Thank you in advance.
Derek
-- Mohammed Naser VEXXHOST, Inc.
participants (2)
-
Derek O keeffe
-
Mohammed Naser