Open Stack

On Mon, Feb 19, 2018 at 08:28:27AM -0500, David Shrewsbury wrote:
> Hi,
> 
> On Sun, Feb 18, 2018 at 10:25 PM, Ian Wienand <iwienand at redhat.com> wrote:
> 
> > Hi,
> >
> > How should we go about restricting certain image builds to specific
> > nodepool builder instances?  My immediate issue is with ARM64 image
> > builds, which I only want to happen on a builder hosted in an ARM64
> > cloud.
> >
> > Currently, the builders go through the image list and check "is the
> > existing image missing or too old, if so, build" [1].  Additionally,
> > all builders share a configuration file [2]; so builders don't know
> > "who they are".
> >
> >
> 
> Why not just split the builder configuration file? I don't see a need to
> add code
> to do this.
> 
In our case (openstack-infra) this will require another change to
puppet-nodepool to support this. Not that we cannot, but it will now mean we'll
have 7[1] different nodepool configuration files to now manage. 4 x
nodepool-launchers, 3 x nodepool-builders, since we have 7 services running.

We could update puppet to start templating or add support for nodepool.d (like
zuul.d) and better split our configs too. I just haven't found time to write
that patch.

I did submit support homing diskimage builds to specific builder[2] a while
back, which is more inline with what ianw is asking. This allows us to assign
images to builders, if set.

[1] http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool
[2] https://review.openstack.org/461239/
> 
> 
> 
> > I'd propose we add an arbitrary tag/match system so that builders can
> > pickup only those builds they mark themselves capable of building?
> >
> > e.g. diskimages would specify required builder tags similar to:
> >
> > ---
> > diskimages:
> >   - name: arm64-ubuntu-xenial
> >     elements:
> >       - block-device-efi
> >       - vm
> >       - ubuntu-minimal
> >       ...
> >     env-vars:
> >       TMPDIR: /opt/dib_tmp
> >       DIB_CHECKSUM: '1'
> >       ...
> >     builder-requires:
> >       architecture: arm64
> > ---
> >
> > The nodepool.yaml would grow another section similar:
> >
> > ---
> > builder-provides:
> >   architecture: arm64
> >   something_else_unique_about_this_buidler: true
> > ---
> >
> > For OpenStack, we would template this section in the config file via
> > puppet in [2], ensuring above that only our theoretical ARM64 build
> > machine had that section in it's config.
> >
> > The nodepool-buidler build loop can then check that its
> > builder-provides section has all the tags specified in an image's
> > "builder-requires" section before deciding to start building.
> >
> > Thoughts welcome :)
> >
> > -i
> >
> > [1] https://git.openstack.org/cgit/openstack-infra/nodepool/
> > tree/nodepool/builder.py#n607
> > [2] https://git.openstack.org/cgit/openstack-infra/project-
> > config/tree/nodepool/nodepool.yaml
> >
> > _______________________________________________
> > Zuul-discuss mailing list
> > Zuul-discuss at lists.zuul-ci.org
> > http://lists.zuul-ci.org/cgi-bin/mailman/listinfo/zuul-discuss
> >
> 
> 
> 
> -- 
> David Shrewsbury (Shrews)

> _______________________________________________
> Zuul-discuss mailing list
> Zuul-discuss at lists.zuul-ci.org
> http://lists.zuul-ci.org/cgi-bin/mailman/listinfo/zuul-discuss