Open Stack

On Fri, 2019-04-12 at 01:23 +0000, Guo, Ruijing wrote:
> Hi, Clark
> 
> Do you have some guide to create CI based on nested virt? We can try it and see how much gap & stable?
the intel nfv ci is an example of this.

assuming you ment how do i creat a third party ci job this is the basic steps.

in my experience it more or less jsut works however the kernel that ships in ubuntu 18.04 is broken
the ubuntu 16.04 kernel works fine as does the centos 7 kernel
if you build it on ubunut install ukuu and use it to install the unpatched mainline 5.0 kernel.

the kernel in the guest is less important then getting the kernel on the host correct.

on the host you obviorly need to also enable kvm nested virt in the kernel module parmaters.
sean at cloud-5:~$ cat /sys/module/kvm_intel/parameters/nested
Y
i belive its on by default as of one of the latest kernel its definetly on by default in 5.0

i enable nested virt via the kernel command line 
sean at cloud-5:~$ cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-5.0.0-050000-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro quiet intel_iommu=on iommu=pt
console=tty0 serial console=ttyS0,115200n8 hugepagesz=2M hugepages=51200 irqaffinity=0,12,24,36 kvm-intel.nested=1
nfs.enable_ino64=1 nohz_full=1-11,13-23,25-35,37-47 numa_balancing=enable rcu_nocbs=1-11,13-23,25-35,37-47 rcu_nocb_poll
vsyscall=native

the rest is the same as setting up any other ci.

if you ment is the a guide to setting up a job that used nested virt instead of how to create a nested virt ci you can
ignore the above and its reltvly trivial. you just need to set the libvirt virt dirver to kvm in you job config the
issue is on most vms upstream that will cause the build to fail as they dont have nested virt enableded on the host.

there are a small set of nodeset that do have nested virt but for now im currently exploing alternitive to nested virt
for ovsdpdk testing. i have 2 paths that i am currently looking at. first the mttcg qemu backend with libvirt 5.2 via
a fedora based job and second qemu with file based memory.

if they prove unviable ill switch my efforts back to nested virt either via openlab or reaching out to the ci providers
that currnetly provied nested virt vms to the gate and asking there permission to use there nodesets for the testing.

regards
sean

> 
> Thanks,
> -Ruijing
> 
> -----Original Message-----
> From: Clark Boylan [mailto:cboylan at sapwetik.org] 
> Sent: Tuesday, April 9, 2019 7:05 AM
> To: openstack-discuss at lists.openstack.org
> Subject: Re: [CI] nested virtualization support in OpenStack CI
> 
> 
> 
> On Mon, Apr 8, 2019, at 3:54 PM, Guo, Ruijing wrote:
> > Hi, Clark
> > 
> > I am trying to add ovsdpdk ci job in neutron. Ovsdpdk need nested 
> > virtualization.
> > 
> 
> Can you describe this requirement in a bit more detail? Looking at docs it appears you need to have large pages. Maybe
> a "picture" will help too:
> 
>   Workload VM
>   (cirros or similar small VM running under
>    emulation or virtualized if nested virt is enabled)
>        |
>       V
>   OpenStack Infra Test Node ("middle VM")
>     This is where ovsdpdk runs right? And this requires large pages?
>        |
>       V
>   Hypervisor (managed by our cloud providers)
> 
> Are we not able to do large pages with emulated workload VMs? Also can we do large pages on the Workload VM if the
> Infra Test Node isn't configured for them? If the infra test node isn't configured for them do we also need support
> for large pages from the Hypervisor?
> 
> If we can do large pages with emulated workload VMs and the hypervisor isn't required to support that we should be
> able to configure large pages on the Infra Test Node and boot emulated guests, but my guess is that at least one of
> these things won't work.
> In any case specific details on what is required and where it is required will be useful.
> 
> Clark
> 
>