Open Stack

On Tue, Aug 08, 2017 at 09:57:24AM -0500, Monty Taylor wrote:
> Heya,
> 
> I started working on the Zuul Migration script - largely because we're
> starting to have more mappings of old job to new job than I can keep in my
> head and I wanted to be able to write them down as we make them.
> 
> https://review.openstack.org/#/q/topic:zuul-v3-migration
> 
> I doesn't do much yet - but I did add a zuul-v3 job that runs the migration
> on project-config and collects the results so we should be able to examine
> that as we work on it.
> 
> I've run in to a conceptual issue already that I'd like feedback on.
> 
> In dealing with project-templates, we have a bit of an issue. Take this
> snippet from layout.yaml:
> 
> project-templates:
>   - name: loci-jobs
>     check:
>       - 'gate-{name}-ubuntu-xenial'
>     gate:
>       - 'gate-{name}-ubuntu-xenial'
> 
> We have two choices of how we can do transforms of things we don't
> explicitly map. We can expand them all and make completely equivilent jobs,
> so that we'd generate the following jobs:
> 
> - job:
>     name: gate-loci-cinder
> 
> - job:
>     name: gate-loci-glance
> 
> If we do that, then we can't really migrate the templates and would have to
> put expanded templates into the resulting project pipeline config, so we'd
> have:
> 
> - project:
>     name: openstack/loci-cinder
>     check:
>       - gate-loci-cinder
>     gate:
>       - gate-loci-cinder
> 
> this is the safest - we KNOW that we can do this migration today and the
> jobs will all work as today - but the resulting config will be uglier. This
> is also a data loss as we'll lose ALL of our current templates.
> 
> We could also depend on the mapping construct and define a mapping for these
> loci jobs:
> 
> - old: gate-{name}-ubuntu-xenial
>   name: test-loci
> 
> in which case we could make:
> 
> - job:
>     name: test-loci
> 
> - project-template:
>     name: loci-jobs
>     check:
>       - test-loci
>     gate:
>       - test-loci
> 
> - project:
>     name: openstack/loci-cinder
>     templates:
>       - loci-jobs
> 
> But doing this will require a much more careful audit of our output - we
> cannot be sure that cases we haven't looked at will work and we essentially
> need to hand-verify every project-template to make sure the jobs it contains
> don't have edge-conditions we need to map.
> 
> As a third option - we could auto-expand {name} in project-templates to the
> name of the project template and generate jobs that way:
> 
> - job:
>     name: gate-loci-jobs
> 
> - project-template:
>     name: loci-jobs
>     check:
>       - gate-loci-jobs
>     gate:
>       - gate-loci-jobs
> 
> - project:
>     name: openstack/loci-cinder
>     templates:
>       - loci-jobs
> 
> This is automatic and doesn't have data loss - but might be a little
> confusing. It also means for some of our templates we'll have duplicate jobs
> we don't need. {name}-tarball, for instance, shows up in many template. Now
> - we'll have a mapping for {name}-tarball - but for jobs that we don't have
> explicit mappings for and that do show up in multiple templates it might be
> weird.

I am fine with option 3, I don't expect it to be 100% perfect but should give us
a good base to start refactoring things. I see this as a 1 time cost, that once
import for openstack-infra, we can start work on reducing duplicate data.



> 
> Thoughts?
> 
> _______________________________________________
> OpenStack-Infra mailing list
> OpenStack-Infra at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra