Open Stack

Are all of your instance_types tables identical across the cell/API DBs?
And also marked is_public?


-AH

On Mon, Nov 23, 2015 at 6:25 PM, Belmiro Moreira <
moreira.belmiro.email.lists at gmail.com> wrote:

> Hi Matt,
> I'm rolling kilo 2015.1.1
>
> After resizing an instance (state: verify_resize) I have the following in
> the instance_extra table:
>
> API cell DB:
> {"new": null, "old": null, "cur": {"nova_object.version": "1.1",
> "nova_object.changes": ["root_gb", "name", "ephemeral_gb", "memory_mb",
> "vcpus", "extra_specs", "swap", "rxtx_factor", "flavorid", "vcpu_weight",
> "id"], "nova_object.name": "Flavor", "nova_object.data": {"root_gb": 20,
> "name": "m1.small", "ephemeral_gb": 0, "memory_mb": 2048, "vcpus": 1,
> "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0, "flavorid": "2",
> "vcpu_weight": null, "id": 5}, "nova_object.namespace": "nova"}}
>
> and in the compute cell DB:
> {"new": {"nova_object.version": "1.1", "nova_object.changes": ["deleted",
> "ephemeral_gb", "updated_at", "disabled", "extra_specs", "rxtx_factor",
> "is_public", "flavorid", "id", "root_gb", "deleted_at", "name",
> "created_at", "memory_mb", "vcpus", "swap", "vcpu_weight"], "
> nova_object.name": "Flavor", "nova_object.data": {"disabled": false,
> "root_gb": 40, "name": "m1.medium", "flavorid": "3", "deleted": false,
> "created_at": null, "ephemeral_gb": 0, "updated_at": null, "memory_mb":
> 4096, "vcpus": 2, "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0,
> "is_public": true, "deleted_at": null, "vcpu_weight": 0, "id": 1},
> "nova_object.namespace": "nova"}, "old": {"nova_object.version": "1.1",
> "nova_object.changes": ["root_gb", "name", "ephemeral_gb", "memory_mb",
> "vcpus", "extra_specs", "swap", "rxtx_factor", "flavorid", "vcpu_weight",
> "id"], "nova_object.name": "Flavor", "nova_object.data": {"root_gb": 20,
> "name": "m1.small", "ephemeral_gb": 0, "memory_mb": 2048, "vcpus": 1,
> "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0, "flavorid": "2",
> "vcpu_weight": null, "id": 5}, "nova_object.namespace": "nova"}, "cur":
> {"nova_object.version": "1.1", "nova_object.changes": ["deleted",
> "ephemeral_gb", "updated_at", "disabled", "extra_specs", "rxtx_factor",
> "is_public", "flavorid", "id", "root_gb", "deleted_at", "name",
> "created_at", "memory_mb", "vcpus", "swap", "vcpu_weight"], "
> nova_object.name": "Flavor", "nova_object.data": {"disabled": false,
> "root_gb": 40, "name": "m1.medium", "flavorid": "3", "deleted": false,
> "created_at": null, "ephemeral_gb": 0, "updated_at": null, "memory_mb":
> 4096, "vcpus": 2, "extra_specs": {}, "swap": 0, "rxtx_factor": 1.0,
> "is_public": true, "deleted_at": null, "vcpu_weight": 0, "id": 1},
> "nova_object.namespace": "nova"}}
>
>
> The quota is calculated in the API cell and the new flavor information is
> missing there.
> I don't see any relevant/related logs using DEBUG.
>
> thanks,
> Belmiro
>
>
> On Sun, Nov 22, 2015 at 1:02 AM, Matt Riedemann <
> mriedem at linux.vnet.ibm.com> wrote:
>
>>
>>
>> On 11/21/2015 3:47 PM, Belmiro Moreira wrote:
>>
>>> Hi,
>>> We are about to upgrade nova to kilo using cells and we noticed
>>> the resize/migrate functionality is not working properly.
>>>
>>> The instance is correctly resized/migrated but fails to
>>> “confirm resize” with the following trace:
>>>
>>> 2015-11-21 22:40:49.804 26786 ERROR nova.api.openstack.wsgi
>>> [req-67f6a225-602c-426c-a33a-02c3f5e3093d belmiro
>>> 40c7729b-c8e5-432f-81d5-a41543c8bf13] Exception handling resource:
>>> 'NoneType' object has no attribute '__getitem__'
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi Traceback
>>> (most recent call last):
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 821,
>>> in _process_stack
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi
>>> action_result = self.dispatch(meth, request, action_args)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 911,
>>> in dispatch
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> method(req=request, **action_args)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py",
>>> line
>>> 823, in _action_confirm_resize
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi
>>> self.compute_api.confirm_resize(context, instance)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 233, in
>>> wrapped
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> func(self, context, target, *args, **kwargs)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 222, in
>>> inner
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> function(self, context, instance, *args, **kwargs)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 250, in
>>> _wrapped
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> fn(self, context, instance, *args, **kwargs)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 203, in
>>> inner
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> f(self, context, instance, *args, **kw)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2452, in
>>> confirm_resize
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     deltas =
>>> self._downsize_quota_delta(context, instance)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2537, in
>>> _downsize_quota_delta
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> API._resize_quota_delta(context, new_flavor, old_flavor, 1, -1)
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2504, in
>>> _resize_quota_delta
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     if
>>> compare * _quota_delta('vcpus') > 0:
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi   File
>>> "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2501, in
>>> _quota_delta
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi     return
>>> sense * (new_flavor[resource] - old_flavor[resource])
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi TypeError:
>>> 'NoneType' object has no attribute '__getitem__'
>>> 2015-11-21 22:40:49.804 26786 TRACE nova.api.openstack.wsgi
>>>
>>>
>>>
>>> I can't reproduce in master so this must have been fixed meanwhile,
>>> however I can't identify the patch.
>>>
>>> Is someone experiencing the same problem? and know the commit that
>>> fixed it?
>>>
>>> thanks,
>>> Belmiro
>>>
>>>
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>
>>>
>> I'm not seeing anything very different in that specific utility code path
>> in liberty or mitaka, so I'd have to assume there is something going on in
>> the online flavor migration when lazy-loading the flavor from the instance
>> object such that the old or new flavor is None.
>>
>> Are you rolling to kilo 2015.1.0 or the latest release 2015.1.2?
>>
>> Are there any debug log messages before the error/trace where it says
>> it's lazy-loading the flavor and doing anything there? If you know the
>> instance in question, you could check the instance_system_metadata and
>> instance_extra tables to see if the old/new flavor information has been
>> migrated successfully.
>>
>> I don't think you'd have this problem in master because the online flavor
>> migration was dropped in liberty (there is a DB migration that stops you
>> from continuing if you haven't manually migrated the flavors yet).
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20151124/38b5e9f1/attachment.html>