Open Stack

On Mon, Apr 8, 2019, at 5:21 AM, Jeremy Stanley wrote:
> On 2019-04-08 00:42:46 +0000 (+0000), Guo, Ruijing wrote:
> > I like Sean's idea. Can we create jobs based on nested KVM &
> > customized OS? If CI is stable enough, we can move non-voting to
> > voting.
> [...]
> 
> There's nothing stopping you (or anyone) from creating such a CI
> system to augment reporting on how specific workloads fair with
> certain nested virt implementations.
> 
> The OpenDev CI system on the other hand is made of a heterogenous
> mix of donations from a variety of service providers with various
> configurations, and even some of those which advertise via CPU flags
> that they offer nested virt acceleration don't provide a stable
> implementation of it. There are already projects opportunistically
> testing with nested virtualization support turned on in their jobs
> to varying degrees of success, so it would be good to find out what
> problems their recent experiences reveal. It's also very much the
> case that even if it's working for a job today, it can start
> crashing jobs randomly tomorrow when a kernel version changes on
> guests or on hosts or we add another service provider into the mix.

Right one of the biggest concerns we have is the lack of observed stability with nested virt. This isn't a theoretical concern. In the not too distant past Tripleo jobs running on CentOS were crashing the "middle" VM and no one noticed until I started looking into job failures. The fix for that required a base hypervisor kernel update which is not something we control.

That said understanding requirements here (which was asked for earlier in the thread, but I don't see a response) is useful when trying to figure out the best way to approach these situations. It is possible there are other reasonable avenues that could be taken.

Clark