Hi, we are currently working on a new environment for our school to teach some basic knowledge about OpenStack. Our goal is to run OpenStack training-labs for each student on VMs running on ESXi. Therefor we've enabled nested virtualization (vhv.enable=TRUE) for the VMs in vSphere. Then we basically do following on Ubuntu 18.04.2: ``` echo 1 > /sys/module/kvm/parameters/ignore_msr # to avoid following error while building the base image: "This usually means the libguestfs appliance failed to start or crashed." sudo apt-get install -y qemu-kvm libvirt-bin libguestfs-tools bridge-utils virt-manager virsh pool-create-as --name default --type dir --target /var/lib/libvirt/images wget http://tarballs.openstack.org/training-labs/dist/labs-stable-rocky.tgz tar xzf labs-stable-rocky.tgz cd labs/ ``` We build the base image ubuntu-18.04-server-amd64 with `./st.py --build basedisk --provider kvm` and then the cluster `./st.py --build cluster --provider kvm`. Everything works fine till...: stacktrain.log: ``` 12256 13:08:19.992 stacktrain.core.autostart INFO Start autostart/04_setup_nova_compute.sh 12256 13:08:19.994 stacktrain.core.ssh DEBUG vm_ssh: ssh -q -i /opt/oslab/labs/lib/osbash-ssh-keys/osbash_key -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ConnectTimeout=20 -o ControlPath=none -p 22 osbash@192.168.122.208 bash autostart/04_setup_nova_compute.sh && rm -vf autostart/04_setup_nova_compute.sh 12256 13:08:19.994 stacktrain.core.ssh DEBUG Writing live log for ssh call at /opt/oslab/labs/log/021_04_setup_nova_compute.auto. 12256 13:08:56.551 stacktrain.core.ssh ERROR ssh returned status 255. 12256 13:08:56.552 stacktrain.core.autostart ERROR Script failure: 04_setup_nova_compute 11272 13:08:57.110 stacktrain.core.autostart ERROR Script failed. Exiting. ``` ssh.log: ``` ssh -q -i /opt/oslab/labs/lib/osbash-ssh-keys/osbash_key -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ConnectTimeout=20 -o ControlPath=none -p 22 osbash@192.168.122.208 mkdir -p autostart ssh -q -i /opt/oslab/labs/lib/osbash-ssh-keys/osbash_key -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ConnectTimeout=20 -o ControlPath=none -p 22 osbash@192.168.122.208 bash autostart/04_setup_nova_compute.sh && rm -vf autostart/04_setup_nova_compute.sh ``` For me it looks like both VMs (controller + compute1) get the same IP address (with `./st.py --build nodes_only --provider kvm`): ... Your cluster nodes: INFO VM name: compute1 INFO SSH login: ssh osbash@192.168.122.47 INFO (password: osbash) INFO VM name: controller INFO SSH login: ssh osbash@192.168.122.47 INFO (password: osbash) INFO Dashboard: Assuming horizon is on controller VM. INFO http://192.168.122.47/horizon/ INFO User : myuser (password: myuser_user_pass) INFO User : admin (password: admin_user_secret) INFO Network: mgmt INFO Network address: 10.0.0.0 INFO Network: provider INFO Network address: 203.0.113.0 ```
From within the VMs:
# controller:
osbash@osbash:~$ ifconfig ens3
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.122.47 netmask 255.255.255.0 broadcast 192.168.122.255
inet6 fe80::5054:ff:fed9:b165 prefixlen 64 scopeid 0x20<link>
ether 52:54:00:d9:b1:65 txqueuelen 1000 (Ethernet)
...
osbash@osbash:~$ netplan ip leases ens3
# This is private data. Do not parse.
ADDRESS=192.168.122.47
NETMASK=255.255.255.0
ROUTER=192.168.122.1
SERVER_ADDRESS=192.168.122.1
NEXT_SERVER=192.168.122.1
BROADCAST=192.168.122.255
T1=1800
T2=3150
LIFETIME=3600
DNS=192.168.122.1
HOSTNAME=osbash
CLIENTID=ffb55e67ff00020000ab11bba91ddef33599c5
osbash@osbash:~$ cat /etc/machine-id
b8c7057a4c084e69aa3743960d41d5ce
# compute1:
osbash@compute1:~$ ifconfig ens3
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.122.47 netmask 255.255.255.0 broadcast 192.168.122.255
inet6 fe80::5054:ff:fe17:b640 prefixlen 64 scopeid 0x20<link>
ether 52:54:00:17:b6:40 txqueuelen 1000 (Ethernet)
...
osbash@compute1:~$ netplan ip leases ens3
# This is private data. Do not parse.
ADDRESS=192.168.122.47
NETMASK=255.255.255.0
ROUTER=192.168.122.1
SERVER_ADDRESS=192.168.122.1
NEXT_SERVER=192.168.122.1
BROADCAST=192.168.122.255
T1=1800
T2=3150
LIFETIME=3600
DNS=192.168.122.1
HOSTNAME=compute1
CLIENTID=ffb55e67ff00020000ab11bba91ddef33599c5
osbash@compute1:~$ cat /etc/machine-id
b8c7057a4c084e69aa3743960d41d5ce
I believe the VMs share the same IP address because of the same machine-id (which is used as DUID: http://manpages.ubuntu.com/manpages/bionic/man5/networkd.conf.5.html) Am I wrong? Thanks for your help, Julian