Open Stack

Hi,

I've been working on reconciling a confusing part of the Nodepool config
file: the mapping between labels, provider images, and diskimages.

Broadly speaking the three constructs are:

diskimages: the file(s) that DIB produces
provider images: a disk image uploaded to a provider, combined with
  flavor information
labels: a set of nodes created from provider images

I think the biggest problem is that provider images are combining two
concepts which should be distinct: uploaded diskimages and flavors used
for launching nodes.  They hold at least one piece of data for each of
these functions: the image metadata (which is used at upload time) and
the flavor (used at node creation time), which makes it impossible to
disentangle them without a change.

Adding to the confusion is simply the use of the word 'images' in
provider images, as it's a little unclear what an image is in this
context (a single diskimage?  one of may images uploaded to the cloud?
an artificial construct combining the two?)

The literal way to reconcile the current configuration syntax would be to
upload the same diskimage to a provider multiple times, each as a
different provider image (with its own set of metadata).  That seems
very wasteful.

Because we did not include support for that (nor should we have, I
think), we needed to take at least one step in the direction of
correcting this situation when making the new zookeeper builders.  To
that end, we merged this change:

  https://review.openstack.org/396749

which removed the diskimage parameter from the provider image stanza.
This made an implicit 1:1 connection between the provider images and
diskimages, based on their having the same name.  That means that we
know which metadata to use when uploading a diskimage to a provider.
However, it also means that there is now not a way to specify more than
one flavor with the same image.  That is not a feature we use currently
(especially since the allocator does not take flavor size into acconut)
but it is one we want to have in the future.

I don't see a way to support both of these features (single upload and
multiple flavors) without a change to the configuration.  I propose that
we keep 396749 in place, and iterate forward.

I think we should change the provider images section to separate out the
parts pertaining to diskimages and those pertaining to flavors.
Something like:

  labels:
    - name: small-ubuntu-trusty
      ready-script: configure_mirror.sh
      min-ready: 1
  
  providers:
    - name: cloud
      diskimages:
        - name: ubuntu-trusty
          metadata:
            foo: bar
          labels:
            - name: small-ubuntu-trusty
              ram: 2g
            - name: large-ubuntu-trusty
              ram: 8g
  
  diskimages:
    - name: ubuntu-trusty
      private-key: /home/nodepool/.ssh/id_rsa
      elements: ...

That also lets us remove the 'providers' section from each label
definition.  That is used to indicate which providers should be used to
create nodes of each label, but by associating labels with provider
copies of diskimages, it is simple to add or remove those label entries
(which would not affect the diskimage entry, whose addition/removal
would cause image uploads or deletions).  I also moved the 'private-key'
attribute to the diskimage section, since that should not differ by
provider.

Does this sound like a reasonable path forward?

Thanks,

Jim