simple build fails to allocate network interface
Hullo I’m trying to spin up a simple OS environment in a single vm and think that I’m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I’ve tried this on a couple of laptops, with the same result, so I suspect that it’s something to do with how I’ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I’ve got to get it working to log any bugs). I’m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try: `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble` The network id is identified from this: “”" [vagrant@localhost devstack]$ openstack network list +--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | +--------------------------------------+---------+——————————————————————————————————————+ “”” Using a public network fails with an error that this isn’t allowed. With the private network above, the failure throws out the following errors. “”" [vagrant@localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2728, in _build_resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] yield resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling. “”" The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): “”" # -*- mode: ruby -*- # vi: set ft=ruby : $script = <<-SHELL dnf update -y #apt update #apt upgrade echo "updated" systemctl disable firewalld systemctl stop firewalld systemctl disable NetworkManager systemctl stop NetworkManager systemctl enable network systemctl start network #dnf config-manager --enable powertools #dnf install -y centos-release-openstack-yoga dnf update -y dnf install -y git python3-pip #dnf install -y openstack-packstack #packstack --provision-demo=n --os-neutron-ml2-type-drivers=vxlan,flat,vlan --gen-answer-file=packstack-answers.txt #sed -i -e 's:10.0.2.15:10.1.0.10:' packstack-answers.txt # not for centos 9 #dnf install -y network-scripts dnf install -y net-tools #cat /vagrant/answers.addon >> packstack-answers.txt # don't want this, but httpd won't start without it setenforce 0 pip3 install invoke #packstack --allinone cd /vagrant # permissions all wrong: ./go.sh # need su - vagrant cd /vagrant && ./go.sh (?) # cd devstack # not working: wrong user ./stack.sh SHELL Vagrant.configure(2) do |config| config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64" # machines = { # 'node1.example.dd' #'node1.example.dd' => { :ip => '10.1.0.10'}, # 'node2.example.dd' => { :ip =>'10.1.0.12'}, # } config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true config.ssh.pty = true config.vm.provision "shell", inline: $script config.vm.network "forwarded_port", guest: 80, host: 8080 # machines.each do | hostname, attrs| # config.vm.define hostname do |machine| # machine.vm.hostname = hostname # machine.vm.network :private_network, :ip => attrs[:ip] # machine.vm.provider "virtualbox" do | v | config.vm.provider "virtualbox" do | v | #v.memory = "4096" #v.memory = "8192" v.memory = "9216" v.cpus = "2" end #end # end end “”” I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files. Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details? Tim
On 9 Jun 2022, at 09:58, tim+openstack.org@coote.org wrote: Hullo I’m trying to spin up a simple OS environment in a single vm and think that I’m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I’ve tried this on a couple of laptops, with the same result, so I suspect that it’s something to do with how I’ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I’ve got to get it working to log any bugs). I’m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try: `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble` The network id is identified from this: “”" [vagrant@localhost devstack]$ openstack network list +--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | +--------------------------------------+---------+——————————————————————————————————————+ “”” Using a public network fails with an error that this isn’t allowed. With the private network above, the failure throws out the following errors. “”" [vagrant@localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): [snip] “”" The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): “”" # -*- mode: ruby -*- # vi: set ft=ruby : [snip] Vagrant.configure(2) do |config| config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64" config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true config.ssh.pty = true config.vm.provision "shell", inline: $script config.vm.network "forwarded_port", guest: 80, host: 8080 config.vm.provider "virtualbox" do | v | v.memory = "9216" v.cpus = "2" end end “”” I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files. Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details? Tim
Hello: This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too. Regards. On Thu, Jun 9, 2022 at 1:53 PM <tim+openstack.org@coote.org> wrote:
On 9 Jun 2022, at 09:58, tim+openstack.org@coote.org wrote:
Hullo
I’m trying to spin up a simple OS environment in a single vm and think that I’m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I’ve tried this on a couple of laptops, with the same result, so I suspect that it’s something to do with how I’ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I’ve got to get it working to log any bugs).
I’m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try:
`openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble`
The network id is identified from this: “”" [vagrant@localhost devstack]$ openstack network list
+--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets |
+--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 |
+--------------------------------------+---------+——————————————————————————————————————+ “””
Using a public network fails with an error that this isn’t allowed.
With the private network above, the failure throws out the following errors. “”" [vagrant@localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last):
[snip] “”"
The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): “”" # -*- mode: ruby -*- # vi: set ft=ruby :
[snip] Vagrant.configure(2) do |config|
config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64"
config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true
config.ssh.pty = true
config.vm.provision "shell", inline: $script
config.vm.network "forwarded_port", guest: 80, host: 8080
config.vm.provider "virtualbox" do | v | v.memory = "9216" v.cpus = "2" end
end “””
I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files.
Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details?
Tim
On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez <ralonsoh@redhat.com> wrote:
Hello:
This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too.
Regards.
Hullo thanks for the response. I don’t know what to look for wrt ml2 driver - I’m using the default `stack.sh` installation script from the devstack repo. The neutron log has helped, I think. I found this useful as a starting point:https://bit.ly/3MG6jnR as it enables log extraction of events relating to the creation of a specific server (although that’s still quite large for a mailing list). I think that this may help with your question (but as I said, I don’t really understand the question). There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp addresses mentioned in the log from the vm. I’ve no idea where they come from, however. Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updated VIF entry in instance network info cache for port 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}} Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updating instance_info_cache with network_info: [{"id": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": "gateway", "version": 6, "meta": {}}, "ips": [{"address": "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], "meta": {"injected": false, "tenant_id": "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5ae4c0d6-2a", "ovs_interfaceid": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=20725) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'localhost.localdomain', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, if_exists=True) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': 'compute:nova', 'neutron:network_name': 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': 'localhost.localdomain', 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': []}, if_exists=False) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for instance with vm_state building and task_state spawning. Event states are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b>
Hi, From your log below you have this: nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b That comes from nova/network/neutron.py: if net.get('router:external') and not net.get('shared'): raise exception.ExternalNetworkAttachForbidden( network_uuid=net['id']) And from your original email 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f is the "public" network, not the "shared" network you created. My guess if you don't have the "shared" attribute set on that network, or you're using the wrong network. What does 'openstack network show 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f' output look like? -Brian On 6/10/22 06:52, tim+openstack.org@coote.org wrote:
On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez <ralonsoh@redhat.com <mailto:ralonsoh@redhat.com>> wrote:
Hello:
This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too.
Regards.
Hullo thanks for the response. I don’t know what to look for wrt ml2 driver - I’m using the default `stack.sh` installation script from the devstack repo.
The neutron log has helped, I think. I found this useful as a starting point:https://bit.ly/3MG6jnR <https://bit.ly/3MG6jnR> as it enables log extraction of events relating to the creation of a specific server (although that’s still quite large for a mailing list).
I think that this may help with your question (but as I said, I don’t really understand the question).
There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp addresses mentioned in the log from the vm. I’ve no idea where they come from, however. Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updated VIF entry in instance network info cache for port 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}} Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updating instance_info_cache with network_info: [{"id": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": "gateway", "version": 6, "meta": {}}, "ips": [{"address": "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], "meta": {"injected": false, "tenant_id": "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5ae4c0d6-2a", "ovs_interfaceid": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=20725) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'localhost.localdomain', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, if_exists=True) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': 'compute:nova', 'neutron:network_name': 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': 'localhost.localdomain', 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': []}, if_exists=False) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for instance with vm_state building and task_state spawning. Event states are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b>
Hi Brian You’re correct about that. The Attachment to public networks means, I think, that it needs to be a private network. In my follow up, I managed to get much more detail about a specific server creation, and showed, I think, that the vm had been trying to use a couple of dhcp servers (v4 and v6), that afaict didn’t exist - I couldn’t ping them and didn’t recognise the network segments. I couldn’t work out where they came from. I got that detail by reviewing this: https://bit.ly/3MG6jnR atm, I’m trying to see if I can get this to work: https://bit.ly/3mD60PU However, I immediately ran into some VirtuaBox changes that invalidated the Vagrantfile. I think that I can fix these. The experience highlights to me the need for a decent test environment. Even then, I suspect that it’s very easy to have some of the latent bugs that you get in distributed systems, which undergrad courses note (https://bit.ly/2OgyWiI: I’m particularly fond of the example that lurks in Cassandra from that lecture set), and which AWS had to resort to formal proofs with TLA+ to even spot. Tim
On 10 Jun 2022, at 14:56, Brian Haley <haleyb.dev@gmail.com> wrote:
Hi,
From your log below you have this:
nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b
That comes from nova/network/neutron.py:
if net.get('router:external') and not net.get('shared'): raise exception.ExternalNetworkAttachForbidden( network_uuid=net['id'])
And from your original email 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f is the "public" network, not the "shared" network you created.
My guess if you don't have the "shared" attribute set on that network, or you're using the wrong network. What does 'openstack network show 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f' output look like?
-Brian
On 6/10/22 06:52, tim+openstack.org@coote.org wrote:
On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez <ralonsoh@redhat.com <mailto:ralonsoh@redhat.com>> wrote:
Hello:
This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too.
Regards.
Hullo thanks for the response. I don’t know what to look for wrt ml2 driver - I’m using the default `stack.sh` installation script from the devstack repo. The neutron log has helped, I think. I found this useful as a starting point:https://bit.ly/3MG6jnR <https://bit.ly/3MG6jnR> as it enables log extraction of events relating to the creation of a specific server (although that’s still quite large for a mailing list). I think that this may help with your question (but as I said, I don’t really understand the question). There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp addresses mentioned in the log from the vm. I’ve no idea where they come from, however. Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updated VIF entry in instance network info cache for port 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}} Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updating instance_info_cache with network_info: [{"id": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": "gateway", "version": 6, "meta": {}}, "ips": [{"address": "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], "meta": {"injected": false, "tenant_id": "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5ae4c0d6-2a", "ovs_interfaceid": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=20725) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'localhost.localdomain', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, if_exists=True) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': 'compute:nova', 'neutron:network_name': 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': 'localhost.localdomain', 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': []}, if_exists=False) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for instance with vm_state building and task_state spawning. Event states are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b>
Hi, Dnia czwartek, 9 czerwca 2022 10:58:46 CEST tim+openstack.org@coote.org pisze:
Hullo I’m trying to spin up a simple OS environment in a single vm and think that I’m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I’ve tried this on a couple of laptops, with the same result, so I suspect that it’s something to do with how I’ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I’ve got to get it working to log any bugs).
I’m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try:
`openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble`
The network id is identified from this: “”" [vagrant@localhost devstack]$ openstack network list +--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | +--------------------------------------+---------+——————————————————————————————————————+ “””
Using a public network fails with an error that this isn’t allowed.
With the private network above, the failure throws out the following errors. “”" [vagrant@localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2728, in _build_resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] yield resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling. “”"
The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): “”" # -*- mode: ruby -*- # vi: set ft=ruby :
$script = <<-SHELL dnf update -y #apt update #apt upgrade echo "updated" systemctl disable firewalld systemctl stop firewalld systemctl disable NetworkManager systemctl stop NetworkManager systemctl enable network systemctl start network #dnf config-manager --enable powertools #dnf install -y centos-release-openstack-yoga dnf update -y dnf install -y git python3-pip #dnf install -y openstack-packstack #packstack --provision-demo=n --os-neutron-ml2-type-drivers=vxlan,flat,vlan --gen-answer-file=packstack-answers.txt #sed -i -e 's:10.0.2.15:10.1.0.10:' packstack-answers.txt # not for centos 9 #dnf install -y network-scripts dnf install -y net-tools #cat /vagrant/answers.addon >> packstack-answers.txt # don't want this, but httpd won't start without it setenforce 0 pip3 install invoke #packstack --allinone cd /vagrant # permissions all wrong: ./go.sh # need su - vagrant cd /vagrant && ./go.sh (?) # cd devstack # not working: wrong user ./stack.sh SHELL
Vagrant.configure(2) do |config|
config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64"
# machines = { # 'node1.example.dd' #'node1.example.dd' => { :ip => '10.1.0.10'}, # 'node2.example.dd' => { :ip =>'10.1.0.12'}, # }
config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true
config.ssh.pty = true
config.vm.provision "shell", inline: $script
config.vm.network "forwarded_port", guest: 80, host: 8080
# machines.each do | hostname, attrs| # config.vm.define hostname do |machine| # machine.vm.hostname = hostname # machine.vm.network :private_network, :ip => attrs[:ip]
# machine.vm.provider "virtualbox" do | v | config.vm.provider "virtualbox" do | v | #v.memory = "4096" #v.memory = "8192" v.memory = "9216" v.cpus = "2" end
#end # end end “””
I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files.
Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details?
Tim
This is some issue in neutron while provisioning port. Please check neutron server and neutron agents logs (depending on You backend it can be neutron-ovs-agent and neutron-dhcp-agent, or in case of OVN most of the things You will find in the neutron-server logs). -- Slawek Kaplonski Principal Software Engineer Red Hat
participants (4)
-
Brian Haley
-
Rodolfo Alonso Hernandez
-
Slawek Kaplonski
-
tim+openstack.org＠coote.org