Hi Melanie, After searching through i found that no service is exposing port 5901. Only when I migrate the instance or do any kind of activity port 5901 is exposed on the said hypervisor node and the vm console is accessible. So, I believe your point 'Something about redoing the port bindings is helping the situation sometimes.' is correct. But how do I troubleshoot this? With regards, Swogat Pradhan On Fri, Jun 30, 2023 at 12:29 AM melanie witt <melwittt@gmail.com> wrote:
On 06/25/23 03:50, Swogat Pradhan wrote:
Hi, After doing a console url show after migration, I am still unable to access the console.
My site consists of 1 central site and 2 DCN sites. Consoles for central and DCN02 are working fine without any issues. But when i am creating an instance for DCN01 the console for the instance is not coming up (attached image for reference).
Today I created 3 different VM's using the same flavor, image, security group, the instances were created in the same compute host. Console was not accessible, so I shelved and unshelved all 3 instances, after which I was able to access the console for 2 of those VM's and am still unable to access the console of the 3rd VM no matter what I do.
Apologies for the delayed reply.
It sounds like there may be some type of problem with regard to the network connection from the novnc console proxy service to the DCN01 site rather than something with the console proxy itself given that things work fine with DCN02 and central. You're seeing "Cannot write data: Broken pipe" in the connection between the compute host and the console proxy showing a connection being broken after being established.
As for why shelve and unshelve sometimes helps, it may be because the network port bindings are updated to inactive during the shelve and then they are updated to active during the unshelve. Something about redoing the port bindings is helping the situation sometimes.
It may be worthwhile to check if there is anything different between the networks/ports the instances with working consoles have vs what the instances with non-working consoles have.
-melwitt
On Sat, Jun 24, 2023 at 2:00 AM melanie witt <melwittt@gmail.com <mailto:melwittt@gmail.com>> wrote:
On 06/22/23 20:07, Swogat Pradhan wrote: > Hi Mel, > Thank you for your response. > I am facing issues with the instance console (vnc) in the openstack > dashboard, Most of the time I shelve the instance and unshelve the > instance to get the console. > But there are some VM's I created which are not working even after > shelve/unshelve. > > I have used the same director to deploy a total of a central and 2 edge > sites. > This issue is happening on a single edge site. > Cold Migration also helps in some situations.
OK, you didn't mention whether requesting a new console 'openstack console url show --vnc <server>' gets you a working console after a migration (or other event where you see the console stop working). I'm trying to determine whether the behavior you're seeing is expected or a bug. After an instance is moved to a different compute node than the one it was on when the console was started, that console is not expected to work anymore. And a new console needs to be started.
Can you give steps for reproducing the issue? Maybe that will provide more clarity.
-melwitt
> On Fri, Jun 23, 2023 at 12:42 AM melanie witt <melwittt@gmail.com <mailto:melwittt@gmail.com> > <mailto:melwittt@gmail.com <mailto:melwittt@gmail.com>>> wrote: > > On 06/22/23 01:08, Swogat Pradhan wrote: > > Hi, > > Please find the below log: > > [root@dcn01-hci-1 libvirt]# cat virtqemud.log > > 2023-06-22 07:40:01.575+0000: 350319: error : > virNetSocketReadWire:1804 > > : End of file while reading data: Input/output error > > 2023-06-22 07:40:01.575+0000: 350319: error : > virNetSocketWriteWire:1844 > > : Cannot write data: Broken pipe > > > > I think this is causing the problem of not getting the instance > console. > > When you say "instance console" are you referring to an interactive > console like VNC or are you talking about the console log for the > instance? > > If it's the interactive console, if you have a console open and then > migrate the instance, that console will not be moved along with the > instance. When a user requests a console, the console proxy service > establishes a connection to the compute host where the instance is > located. The proxy doesn't know when an instance has been moved though, > so if the instance is moved, the user will need to request a new > console > (which will establish a new connection to the new compute host). > > Is that the behavior you are seeing? > > -melwitt > > > On Fri, Jun 2, 2023 at 11:27 AM Swogat Pradhan > > <swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com> <mailto:swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com>> > <mailto:swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com> > <mailto:swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com>>>> wrote: > > > > Update: > > If the i am performing any activity like migration or resize > of an > > instance whose console is accessible, the console becomes > > inaccessible giving out the following error : something went > wrong, > > connection is closed > > > > The was 1 other instance whose console was not accessible and > i did > > a shelve and unshelve and suddenly the instance console became > > accessible. > > > > This is a peculiar behavior and i don't understand where is > the issue . > > > > With regards, > > Swogat Pradhan > > > > On Fri, Jun 2, 2023 at 11:19 AM Swogat Pradhan > > <swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com> <mailto:swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com>> > <mailto:swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com> > <mailto:swogatpradhan22@gmail.com <mailto:swogatpradhan22@gmail.com>>>> wrote: > > > > Hi, > > I am creating instances in my DCN site and i am unable to get > > the console sometimes, error: something went wrong, > connection > > is closed > > > > I have 3 instances now running on my hci02 node and there is > > console access on 1 of the vm's and the rest two i am not > > getting the console, i have used the same flavor, same image > > same security group for the VM's. > > > > Please suggest what can be done. > > > > With regards, > > Swogat Pradhan > > >