[ops] Running VMware atop of OpenStack (eg: esxi)
Greetings, I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack. Bascially, looking for humans to bounce questions off and see who else is doing it. Thanks! Paul
On Fri, Nov 22, 2019 at 12:21:08PM -0500, Paul Belanger wrote:
Greetings,
I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack.
Bascially, looking for humans to bounce questions off and see who else is doing it.
To avoid confusion, in this context it would be booting esxi a top of kvm (via openstack) not running esxi as its own hypervisor alongside kvm. Thanks! Paul
On Fri, Nov 22, 2019 at 12:39 PM Paul Belanger <pabelanger@redhat.com> wrote:
On Fri, Nov 22, 2019 at 12:21:08PM -0500, Paul Belanger wrote:
Greetings,
I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack.
Bascially, looking for humans to bounce questions off and see who else is doing it.
To avoid confusion, in this context it would be booting esxi a top of kvm (via openstack) not running esxi as its own hypervisor alongside kvm.
I've never done it through openstack but I have ran esxi on the top of kvm many times. You have to ensure kvm is passing the hypervisor thing (vt-x/whatever) to the guest. I think https://libvirt.org/formatdomain.html#elementsFeatures has th eoptiong you need to turn on.
Thanks! Paul
Hi, On Fri, Nov 22, 2019 at 12:21 PM Paul Belanger <pabelanger@redhat.com> wrote:
Greetings,
I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack.
Bascially, looking for humans to bounce questions off and see who else is doing it.
We don't plan on running ESXi on top of KVM but alongside KVM on a baremetal. So I won't be able to address that specific use case. However I can share with you what we are planning to do with config-drive support. (we are actively working on it as I'm typing) At first, we tried to package Glean for VMware and install that package when we built the image. The issue with that is that the package isn't signed. This can affect support and prevent the overall system from being updated due to the presence of an unsigned package. You would need some kind of partnership with VMware to get it signed. We therefore switched to a firstboot script which can be configured in the kickstart when building the image: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.upgrade.doc/GU... (see %firstboot section) You can have multiple %firstboot section both using busybox or python as an interpreter. We have multiple sections performing various steps: 1) Find the config-drive partition. Since it's baremetal, config-drive is a primary partition at the end of the disk. 2) Load the iso9660 module 3) Mount the config-drive 4) A whole Python section which parses config-drive and perform hostname/network/password/publickeys configuration. 5) General cleanup: unmount config-drive, unload iso9660 module Now since it's a firstboot script, VMware no longer complains about unsigned packages. I hope this help. -- Mathieu
On Fri, Nov 22, 2019 at 12:54:50PM -0500, Mathieu Gagné wrote:
Hi,
On Fri, Nov 22, 2019 at 12:21 PM Paul Belanger <pabelanger@redhat.com> wrote:
Greetings,
I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack.
Bascially, looking for humans to bounce questions off and see who else is doing it.
We don't plan on running ESXi on top of KVM but alongside KVM on a baremetal. So I won't be able to address that specific use case.
However I can share with you what we are planning to do with config-drive support. (we are actively working on it as I'm typing)
At first, we tried to package Glean for VMware and install that package when we built the image. The issue with that is that the package isn't signed. This can affect support and prevent the overall system from being updated due to the presence of an unsigned package. You would need some kind of partnership with VMware to get it signed.
We therefore switched to a firstboot script which can be configured in the kickstart when building the image: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.upgrade.doc/GU... (see %firstboot section)
You can have multiple %firstboot section both using busybox or python as an interpreter.
We have multiple sections performing various steps: 1) Find the config-drive partition. Since it's baremetal, config-drive is a primary partition at the end of the disk. 2) Load the iso9660 module 3) Mount the config-drive 4) A whole Python section which parses config-drive and perform hostname/network/password/publickeys configuration. 5) General cleanup: unmount config-drive, unload iso9660 module
Now since it's a firstboot script, VMware no longer complains about unsigned packages.
I hope this help.
Nice, this is exactly the type of thing I was hoping for! Awesome! In fact, my initial though too was also update glean adding vmware support, good to know somebody else tried this first. As for first boot, this too in the approach we are taking. A teammate has created a python script to do the same. My first question was, why couldn't that script be glean? That is the part I am a little confused about. Also, are you interested in working on this 'python' script together upstream? I suspect we might be working to solve the same issue related to hostname, network, SSH. Paul
-- Mathieu
On Fri, Nov 22, 2019 at 1:03 PM Paul Belanger <pabelanger@redhat.com> wrote:
On Fri, Nov 22, 2019 at 12:54:50PM -0500, Mathieu Gagné wrote:
Hi,
On Fri, Nov 22, 2019 at 12:21 PM Paul Belanger <pabelanger@redhat.com> wrote:
Greetings,
I wanted to ask if anybody in the community is running VMware a top of OpenStack in any capacity? In Ansible we are doing a POC as part of our testing platform and running into some common ops issue. For example, what do people usually do for things like using config-drive? Is there any specific tooling you us to customize esxi images to run atop OpenStack.
Bascially, looking for humans to bounce questions off and see who else is doing it.
We don't plan on running ESXi on top of KVM but alongside KVM on a baremetal. So I won't be able to address that specific use case.
However I can share with you what we are planning to do with config-drive support. (we are actively working on it as I'm typing)
At first, we tried to package Glean for VMware and install that package when we built the image. The issue with that is that the package isn't signed. This can affect support and prevent the overall system from being updated due to the presence of an unsigned package. You would need some kind of partnership with VMware to get it signed.
We therefore switched to a firstboot script which can be configured in the kickstart when building the image: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.upgrade.doc/GU... (see %firstboot section)
You can have multiple %firstboot section both using busybox or python as an interpreter.
We have multiple sections performing various steps: 1) Find the config-drive partition. Since it's baremetal, config-drive is a primary partition at the end of the disk. 2) Load the iso9660 module 3) Mount the config-drive 4) A whole Python section which parses config-drive and perform hostname/network/password/publickeys configuration. 5) General cleanup: unmount config-drive, unload iso9660 module
Now since it's a firstboot script, VMware no longer complains about unsigned packages.
I hope this help.
Nice, this is exactly the type of thing I was hoping for! Awesome! In fact, my initial though too was also update glean adding vmware support, good to know somebody else tried this first.
As for first boot, this too in the approach we are taking. A teammate has created a python script to do the same. My first question was, why couldn't that script be glean? That is the part I am a little confused about.
To install Glean, you would usually create a .vib package and install that. But since the package isn't signed, VMware complains. This isn't good if you want to have support from VMware. That's why we are now using a firstboot script which is added and executed from the kickstart, avoiding this whole package signing thing.
Also, are you interested in working on this 'python' script together upstream? I suspect we might be working to solve the same issue related to hostname, network, SSH.
I'm happy to help and share what we have done so far. I however don't have a lot of bandwidth. I used to work on that internal project months ago, that's why I still have some knowledge about the subject. A coworker took over it and I'm helping him whenever I can to answer his questions. He is supposed to work on it this week so I should be able to get more material to share in the following days. -- Mathieu
participants (3)
-
Mathieu Gagné
-
Mauricio Tavares
-
Paul Belanger