[openstack-dev] [nova][vmware] HA and image caching

Matthew Booth mbooth at redhat.com
Thu Feb 19 17:15:56 UTC 2015


I was discussing the problems of configuring active/passive HA earlier
where the active and passive nodes have a different `host` configured.
There are lots of problems, as I've mentioned before, but Gary pointed
out that it probably affects image cache management, too. This had the
potential for unsubtle breakage, so I decided to check it out in a bit
more detail. The bug is real, but fortunately doesn't delete anything
(which is itself a bug):

  storage_users.register_storage_use(CONF.instances_path, CONF.host)
  nodes = storage_users.get_storage_users(CONF.instances_path)

This bit of code assumes shared storage between different nova nodes
which access the same images. We don't have that, so `nodes` above is
going to contain only the current node.

  filters = {'deleted': False,
             'soft_deleted': True,
             'host': nodes}
  filtered_instances = objects.InstanceList.get_by_filters(context,
                           filters, expected_attrs=[], use_slave=True)

We're getting a list of instances which were created by the current node
only.

  self.driver.manage_image_cache(context, filtered_instances)

Which we're passing to the driver.

So, the driver is only going to consider images attached to instances
which were created by the current node. The effect of this is that just
after a switchover, cache management will not consider any instances,
and therefore nothing will be expired. This is still bad, but it's not
going to delete users' stuff.

Incidentally, I remain convinced that we can safely configure
active/passive HA if we configure all nodes with the same `host`. That
way we wouldn't have to worry about all these edge cases, and we
wouldn't need to eventually create a cleanup job to consolidate all
instances running on a single hypervisor to have the same 'host'.

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490



More information about the OpenStack-dev mailing list