[Openstack-operators] [glance] [nova] Glance bug with Kilo upgrade & Nova
clayton at oneill.net
Mon Jun 8 15:46:19 UTC 2015
We tested testing Kilo upgrades in our hardware dev environments last week
and the second time through ran into this bug which right now is probably a
show-stopper for us.
The issue here is that the v1 Glance API allows you to create images with
properties that are 'NULL' in the Glance database. For example:
glance image-create --name cirros_test --disk-format qcow2
--container-format bare --file cirros-0.3.4-x86_64-disk.img --is-public
True --is-protected True --progress --property description=
It's apparently also fairly easy to end up with a NULL description when
editing images properties via Horizon.
The issue is that the v2 Glance API returns these NULL properties to the
client, which then validates them against the schema returned by the v2
API. This schema returns specifies that the description property *must* be
In the Kilo release, Nova has been changed to use the v2 API, so suddenly
this matters. The net effect is that end users can pretty easily create
properties with NULL values, and then won't be able to boot instances using
those images. What makes this worse is that it's completely opaque to end
users, since this just reports that no node was available to schedule the
However, Nova *only* uses the v2 api to list images if the
glance.allowed_direct_url_schemes config key is set in the config file.
However, this config item defaults to an empty array, meaning that by
default it's *always* set. There doesn't appear to be a way to unset a
value with oslo-config that has a default value, blocking off that route to
work around the issue. Disabling the v2 Glance API we don't think will
work, since Nova appears to assume the v2 API is available.
Another work around we've looked at is to change the DB schema for image
properties (yuck) to not allow NULL values. This results in Glance
returning a 500 error since glance-api is attempting to insert an invalid
value. This is better than instances failing in an opaque fashion, but
still pretty horrible.
Has anyone else run into this issue yet? Are there other work arounds that
we've not thought of other than "Don't create images with NULL properties?"
User education is definitely an option, but given the failure mode, it's
not a great solution for us.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-operators