[rdo][ussuri][TripleO][nova][kvm] libvirt.libvirtError: internal error: process exited while connecting to monitor
Hi all. I have finally deployed Ussuri, it went quite good, except some my site specific issues and newton config files remake, but now I face issues when launching an instance. [4] Here are logs from compute: /var/log/containers/nova/nova-compute.log [5] from the controller - nova-conductor.log According to errors from compute, something is wrong with kvm? some older bugs refer to kvm exec permissions [2] a bit more recent [1] which refers to selinux issues, but I have setenforce 0 and tried to create instance, not much changed. Also, [3] very interesting kvm.conf file updates suggested, but I am not sure, how to restart service since config changed, as I am BRAND new to container concept with docker/podman, bit more used to uml :) so, how to reload service in container (when container is running as non-root user)? [1] https://bugzilla.redhat.com/show_bug.cgi?id=1551651 # related to RHOSP7 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1091710 # 6 year old. [3] https://ask.openstack.org/en/question/95198/could-not-access-kvm-kernel-modu... [4] http://paste.openstack.org/show/pBVlVbaQGNuoRLHozHGO/ [5] http://paste.openstack.org/show/OqvozreN5aW4HRXM4g4p/ -- Ruslanas Gžibovskis +370 6030 7030
On Wed, Jun 24, 2020 at 9:12 AM Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
Hi all.
I have finally deployed Ussuri, it went quite good, except some my site specific issues and newton config files remake, but now I face issues when launching an instance.
[4] Here are logs from compute: /var/log/containers/nova/nova-compute.log [5] from the controller - nova-conductor.log
According to errors from compute, something is wrong with kvm? some older bugs refer to kvm exec permissions [2] a bit more recent [1] which refers to selinux issues, but I have setenforce 0 and tried to create instance, not much changed.
Compute nodes are baremetal or virtualized?, I've seen similar bug reports when using nested virtualization in other OSes.
Also, [3] very interesting kvm.conf file updates suggested, but I am not sure, how to restart service since config changed, as I am BRAND new to container concept with docker/podman, bit more used to uml :) so, how to reload service in container (when container is running as non-root user)?
When using podman, the recommended way to restart containers is using systemd: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployme...
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1551651 # related to RHOSP7 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1091710 # 6 year old. [3] https://ask.openstack.org/en/question/95198/could-not-access-kvm-kernel-modu... [4] http://paste.openstack.org/show/pBVlVbaQGNuoRLHozHGO/ [5] http://paste.openstack.org/show/OqvozreN5aW4HRXM4g4p/
-- Ruslanas Gžibovskis +370 6030 7030 _______________________________________________ users mailing list users@lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/users
To unsubscribe: users-unsubscribe@lists.rdoproject.org
Hi Alfredo, Compute nodes are baremetal or virtualized?, I've seen similar bug reports
when using nested virtualization in other OSes.
baremetal. Dell R630 if to be VERY precise.
When using podman, the recommended way to restart containers is using systemd:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployme...
Thank you, I will try. I also modified a file, and it looked like it relaunched podman container once config was changed. Either way, if I understand Linux config correctly, the default value for user and group is root, if commented out: #user = "root" #group = "root" also in some logs, I saw, that it detected, that it is not AMD CPU :) and it is really not AMD CPU. Just for fun, it might be important, here is how my node info looks. ComputeS01Parameters: NovaReservedHostMemory: 16384 KernelArgs: "crashkernel=no rhgb" ComputeS01ExtraConfig: nova::cpu_allocation_ratio: 4.0 nova::compute::libvirt::rx_queue_size: 1024 nova::compute::libvirt::tx_queue_size: 1024 nova::compute::resume_guests_state_on_host_boot: true
Hi Ruslanas! Is it possible to get all logs under /var/log/containers somehow? Thanks! On Wed, Jun 24, 2020 at 2:18 AM Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
Hi Alfredo,
Compute nodes are baremetal or virtualized?, I've seen similar bug reports
when using nested virtualization in other OSes.
baremetal. Dell R630 if to be VERY precise.
When using podman, the recommended way to restart containers is using systemd:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployme...
Thank you, I will try. I also modified a file, and it looked like it relaunched podman container once config was changed. Either way, if I understand Linux config correctly, the default value for user and group is root, if commented out: #user = "root" #group = "root"
also in some logs, I saw, that it detected, that it is not AMD CPU :) and it is really not AMD CPU.
Just for fun, it might be important, here is how my node info looks. ComputeS01Parameters: NovaReservedHostMemory: 16384 KernelArgs: "crashkernel=no rhgb" ComputeS01ExtraConfig: nova::cpu_allocation_ratio: 4.0 nova::compute::libvirt::rx_queue_size: 1024 nova::compute::libvirt::tx_queue_size: 1024 nova::compute::resume_guests_state_on_host_boot: true _______________________________________________ users mailing list users@lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/users
To unsubscribe: users-unsubscribe@lists.rdoproject.org
yes, will check and add it later, once back home. On Wed, 24 Jun 2020, 21:02 Arkady Shtempler, <ashtempl@redhat.com> wrote:
Hi Ruslanas!
Is it possible to get all logs under /var/log/containers somehow?
Thanks!
On Wed, Jun 24, 2020 at 2:18 AM Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
Hi Alfredo,
Compute nodes are baremetal or virtualized?, I've seen similar bug
reports when using nested virtualization in other OSes.
baremetal. Dell R630 if to be VERY precise.
When using podman, the recommended way to restart containers is using systemd:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployme...
Thank you, I will try. I also modified a file, and it looked like it relaunched podman container once config was changed. Either way, if I understand Linux config correctly, the default value for user and group is root, if commented out: #user = "root" #group = "root"
also in some logs, I saw, that it detected, that it is not AMD CPU :) and it is really not AMD CPU.
Just for fun, it might be important, here is how my node info looks. ComputeS01Parameters: NovaReservedHostMemory: 16384 KernelArgs: "crashkernel=no rhgb" ComputeS01ExtraConfig: nova::cpu_allocation_ratio: 4.0 nova::compute::libvirt::rx_queue_size: 1024 nova::compute::libvirt::tx_queue_size: 1024 nova::compute::resume_guests_state_on_host_boot: true _______________________________________________ users mailing list users@lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/users
To unsubscribe: users-unsubscribe@lists.rdoproject.org
hi all, during redeployment, faced this issue now [1] it looks like, on compute deployment step, compute, cannot communicate to VIM. even with a regular ping I can reach it. with regular curl http://IP:5000/v3 I get json: [root@rem0te-compr-0 heat-admin]# podman exec -it c3515b7d46fe curl http://10.120.129.202:5000/v3 {"version": {"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": " http://10.120.129.202:5000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}} [root@rem0te-compr-0 heat-admin]# Also I see in TCP dump, that I receive reply, and even on compute, I see reply coming in.... I am lost. Any ideas? I am using L3 routed networks [2] And these OSP deployment files: [3] [1] http://paste.openstack.org/show/coo2bB418Ik1uiWjEcPn/ [2] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features... [3] https://github.com/qw3r3wq/homelab/tree/master/overcloud On Wed, 24 Jun 2020 at 20:06, Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
yes, will check and add it later, once back home.
On Wed, 24 Jun 2020, 21:02 Arkady Shtempler, <ashtempl@redhat.com> wrote:
Hi Ruslanas!
Is it possible to get all logs under /var/log/containers somehow?
Thanks!
On Wed, Jun 24, 2020 at 2:18 AM Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
Hi Alfredo,
Compute nodes are baremetal or virtualized?, I've seen similar bug
reports when using nested virtualization in other OSes.
baremetal. Dell R630 if to be VERY precise.
When using podman, the recommended way to restart containers is using systemd:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployme...
Thank you, I will try. I also modified a file, and it looked like it relaunched podman container once config was changed. Either way, if I understand Linux config correctly, the default value for user and group is root, if commented out: #user = "root" #group = "root"
also in some logs, I saw, that it detected, that it is not AMD CPU :) and it is really not AMD CPU.
Just for fun, it might be important, here is how my node info looks. ComputeS01Parameters: NovaReservedHostMemory: 16384 KernelArgs: "crashkernel=no rhgb" ComputeS01ExtraConfig: nova::cpu_allocation_ratio: 4.0 nova::compute::libvirt::rx_queue_size: 1024 nova::compute::libvirt::tx_queue_size: 1024 nova::compute::resume_guests_state_on_host_boot: true _______________________________________________ users mailing list users@lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/users
To unsubscribe: users-unsubscribe@lists.rdoproject.org
-- Ruslanas Gžibovskis +370 6030 7030
hi all, I am back, had some issues with MTU. Now looks good, at least deployment part. So I have installed back what I had, and still failing at same point as in first message. I have tried to use: LogTool, how to use it? well, I launched it, but it always say [0] detailed output: File "./PyTool.py", line 596, in <module> random_node=random.choice(overcloud_nodes) I do not get, how to make it work, should it get from stackrc ? as I see in overcloud_nodes = [] all_nodes = exec_command_line_command('source ' + source_rc_file_path + 'stackrc;openstack server list -f json')[ [0] http://paste.openstack.org/show/795345/ On Wed, 24 Jun 2020 at 20:02, Arkady Shtempler <ashtempl@redhat.com> wrote:
Hi Ruslanas!
Is it possible to get all logs under /var/log/containers somehow?
Thanks!
On Wed, Jun 24, 2020 at 2:18 AM Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
Hi Alfredo,
Compute nodes are baremetal or virtualized?, I've seen similar bug
reports when using nested virtualization in other OSes.
baremetal. Dell R630 if to be VERY precise.
When using podman, the recommended way to restart containers is using systemd:
https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployme...
Thank you, I will try. I also modified a file, and it looked like it relaunched podman container once config was changed. Either way, if I understand Linux config correctly, the default value for user and group is root, if commented out: #user = "root" #group = "root"
also in some logs, I saw, that it detected, that it is not AMD CPU :) and it is really not AMD CPU.
Just for fun, it might be important, here is how my node info looks. ComputeS01Parameters: NovaReservedHostMemory: 16384 KernelArgs: "crashkernel=no rhgb" ComputeS01ExtraConfig: nova::cpu_allocation_ratio: 4.0 nova::compute::libvirt::rx_queue_size: 1024 nova::compute::libvirt::tx_queue_size: 1024 nova::compute::resume_guests_state_on_host_boot: true _______________________________________________ users mailing list users@lists.rdoproject.org http://lists.rdoproject.org/mailman/listinfo/users
To unsubscribe: users-unsubscribe@lists.rdoproject.org
-- Ruslanas Gžibovskis +370 6030 7030
participants (3)
-
Alfredo Moralejo Alonso
-
Arkady Shtempler
-
Ruslanas Gžibovskis