Open Stack

On 7/20/2016 7:01 PM, Nikhil Komawar wrote:
> Thanks Matt. The bug looked very descriptive so, I commented my thoughts
> there https://bugs.launchpad.net/python-glanceclient/+bug/1596602
>
> On 7/20/16 4:18 PM, Matt Riedemann wrote:
>> On 7/18/2016 4:41 AM, Erno Kuvaja wrote:
>>> On Mon, Jul 18, 2016 at 5:19 AM, Nikhil Komawar
>>> <nik.komawar at gmail.com> wrote:
>>>> Thanks Matt. I've scheduled for a release of the client this week.
>>>>
>>>> On 7/16/16 4:09 AM, Matt Riedemann wrote:
>>>>> This is more of a heads up than anything.
>>>>>
>>>>> Our internal CI is running Tempest with images that don't have
>>>>> kernel_id or ramdisk_id properties set.
>>>>>
>>>>> We're running from master so nova defaults to use_glance_v1=False.
>>>>>
>>>>> Because of this:
>>>>>
>>>>> https://github.com/openstack/nova/blob/47358449d359a287d21426b4e1f18479a4d1fd36/nova/compute/api.py#L867-L868
>>>>>
>>>>>
>>>>>
>>>>> and this:
>>>>>
>>>>> https://github.com/openstack/nova/blob/47358449d359a287d21426b4e1f18479a4d1fd36/nova/image/glance.py#L835
>>>>>
>>>>>
>>>>>
>>>>> The snapshot image properties get kernel_id and ramdisk_id set to None
>>>>> since that's what the glance v2 schema requires.
>>>>>
>>>>> However, python-glanceclient has it's own outdated copy of the schema
>>>>> which doesn't allow null values for those properties, see bug:
>>>>>
>>>>> https://bugs.launchpad.net/python-glanceclient/+bug/1596602
>>>>>
>>>>> We don't hit this in the community CI because the image that Tempest
>>>>> uses from devstack has the kernel_id and ramdisk_id properties set:
>>>>>
>>>>> http://logs.openstack.org/52/335152/1/check/gate-tempest-dsvm-neutron-src-python-glanceclient/d393db9/logs/devstacklog.txt.gz#_2016-06-28_18_40_12_429
>>>>>
>>>>>
>>>>>
>>>>> But for anyone else upgrading to Newton that has images without those
>>>>> properties set and doesn't have use_glance_v1=True in nova.conf is
>>>>> going to be broken.
>>>>>
>>>>> Since we really want to get people off glance v1 and move to
>>>>> deprecation in Ocata, we need to get this merged and released:
>>>>>
>>>>> https://review.openstack.org/#/c/335152/
>>>>>
>>>>> And bump the minimum required python-glanceclient in
>>>>> global-requirements for Newton.
>>>>>
>>>>> I'm not really sure why python-glanceclient even has it's own copy of
>>>>> the image schema, that seems redundant and error prone given the
>>>>> glance API already validates that, but it's kind of beside the point
>>>>> right now.
>>>>>
>>>>
>>>> --
>>>>
>>>> Thanks,
>>>> Nikhil
>>>>
>>>>
>>>> __________________________________________________________________________
>>>>
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> Hi Matt,
>>>
>>> Something has broken on the way if that shipped schema actually affect
>>> your operation. That should be used _only_ when the client is called
>>> without connection to the Glance API (for example in a case you have
>>> no env set and you call `glance help`), this is btw the reason why we
>>> have that schema shipped with the client.
>>>
>>> As soon as you execute a call that actually interacts with Glance API
>>> we should be pulling the schema from there. So if this is not the
>>> case, we've broken somewhere on the way. I'll try to find time to have
>>> a look.
>>>
>>> - Erno
>>>
>>> __________________________________________________________________________
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> Yeah I guess the schema fix in glanceclient didn't resolve the issue,
>> but I haven't had the time to dig into the bug since we're doing the
>> midcycle this week.
>>
>

For anyone that cares, we figured out the problem. In our case, the 
issue was we didn't have /etc/glance/schema-image.json in the 
deployment. We're also using full disk images so no kernel_id or 
ramdisk_id set on the image (or the snapshot of the instance).

devstack masks all of this because it sets up 
/etc/glance/schema-image.json and it uses a multipart uec cirros image 
by default, so kernel_id and ramdisk_id are set.

The amount of schema configuration that is possible scares me, given the 
glance config option for allow_additional_image_properties (defaults to 
true, but if false, you can't even set kernel_id/ramdisk_id and 
snapshots would be broken with the libvirt driver and a uec image).

And /etc/glance/schema-image.json is configuration, so it might not be 
there, or it might be different.

Nova was passing on assumptions made with the defaults, and how devstack 
/ our CI system works, but Nova as a client of glance isn't actually 
checking the image schema available when creating images, so we could 
potentially hit other issues in the future.

Mike Fedosin has the simple fix for the existing bug though:

https://review.openstack.org/#/c/347971

And it looks like the nova API schema prevents being able to create a 
snapshot with metadata that uses null values, so there is some 
additional protection in that case.

-- 

Thanks,

Matt Riedemann