Open Stack

Hi everyone, happy Monday :)

I'd like to start the discussion about CI-testing the optional 
composable services in the CI (I'm primarily interested in Ironic, but I 
know there are a lot more).

Currently every time we change something in an optional service, we have 
to create a DO-NOT-MERGE patch making the service in question not 
optional. This approach has several problems:

1. It's not usually done for global refactorings.

2. The current CI does not have any specific tests to check that the 
services in question actually works at all (e.g. in my experience the CI 
was green even though nova-compute could not reach ironic).

3. If something breaks, it's hard to track the problem down to a 
specific patch, as there is no history of gate runs.

4. It does not test the environment files we provide for enabling the 
service.

So, are there any plans to start covering optional services? Maybe at 
least a non-HA job with all environment files included? It would be cool 
to also somehow provide additional checks though. Or, in case of ironic, 
to disable the regular nova compute, so that the ping test runs on an 
ironic instance.

WDYT?