Open Stack

On 17.05.17 22:18, Zane Bitter wrote:
> On 16/05/17 10:32, Lance Haig wrote:
>>> What if instead of a directory per release, we just had a 'deprecated'
>>> directory where we move stuff that is going away (e.g. anything
>>> relying on OS::Glance::Image), and then deleted them when it
>>> disappeared from any supported release (e.g. LBaaSv1 must be close if
>>> it isn't gone already).
>>>
>> I agree in general this would be good. How would we deal with users who
>> are running older versions of openstack?
>> Most of the customers I support have Liberty and newer so I would
>> perhaps like to have these available as tested.
>> The challenge for us is that the newer the OStack version the more
>> features are available e.g. conditionals etc..
>> To support that in a backwards compatible fashion is going to be tough I
>> think. Unless I am missing something.
>
> 'stable' branches could achieve that, and it's the most feasible way 
> to actually CI test them against older releases anyway.
Ok this sounds good. We just need to see if it is feasible to implement.
>
>>>> As we've proven, maintaining these templates has been a challenge
>>>> given the
>>>> available resources, so I guess I'm still in favor of not duplicating
>>>> a bunch
>>>> of templates, e.g perhaps we could focus on a target of CI testing
>>>> templates on the current stable release as a first step?
>>>
>>> I'd rather do CI against Heat master, I think, but yeah that sounds
>>> like the first step. Note that if we're doing CI on old stuff then
>>> we'd need to do heat-templates stable branches rather than
>>> directory-per-release.
>>>
>>> With my suggestion above, we could just not check anything in the
>>> 'deprecated' directory maybe?
>> I agree in part.
>> If we are using the heat examples to test the functionality of the
>> master branch then that would be a good idea.
>> If we want to provide useable templates for users to reference and use
>> then I would suggest we test against stable.
>
> The downside of that is you can't add a template that uses a new 
> feature in Heat until after the next release (which includes that 
> feature).
>
> I think the answer here is to have stable heat-templates test against 
> stable heat and master against master.
Agree completely.
>
>> I am sure we could find a way to do both.
>> I would suggets that we first get reliable CICD running on the current
>> templates and fix what we can in there.
>> Then we can look at what would be a good way forward.
>>
>> I am just brain dumping so any other ideas would also be good.
>>>
>>>>> As you guys mentioned in our discussions the Networking example I
>>>>> quoted is
>>>>> not something you guys can deal with as the source project affects
>>>>> this.
>>>>>
>>>>> Unless we can use this exercise to test these and fix them then I am
>>>>> happier.
>>>>>
>>>>> My vision would be to have a set of templates and examples that are
>>>>> tested
>>>>> regularly against a running OS deployment so that we can make sure 
>>>>> the
>>>>> combinations still run. I am sure we can agree on a way to do this
>>>>> with CICD
>>>>> so that we test the fetureset.
>>>>
>>>> Agreed, getting the approach to testing agreed seems like the first
>>>> step -
>>>> FYI we do already have automated scenario tests in the main heat tree
>>>> that
>>>> consume templates similar to many of the examples:
>>>>
>>>> https://github.com/openstack/heat/tree/master/heat_integrationtests/scenario 
>>>>
>>>>
>>>>
>>>> So, in theory, getting a similar test running on heat_templates
>>>> should be
>>>> fairly simple, but getting all the existing templates working is
>>>> likely to
>>>> be a bigger challenge.
>>>
>>> Even if we just ran the 'template validate' command on them to check
>>> that all of the resource types & properties still exist, that would be
>>> pretty helpful. It'd catch of of the times when we break backwards
>>> compatibility so we can decide to either fix it or deprecate/remove
>>> the obsolete template. (Note that you still need all of the services
>>> installed, or at least endpoints in the catalog, for the validation to
>>> work.)
>
> So apparently Thomas already implemented this (nice!). The limitation 
> is that we're ignoring errors where the template contains a resource 
> where the endpoint is not in the service catalog. That's likely to 
> mean a lot of these templates aren't _really_ getting tested.
>
> In theory all we have to do to fix that is add endpoints for all of 
> them in the catalog (afaik we shouldn't need to actually run any of 
> the services).

If we have the ability to have "offline" endpoints to validate against 
this would make CICD for the templates much easier.
It would be good then to have this available for people who are 
developing on a laptop when offline.

I hope we are able to do this.

Lance