Open Stack

-- Henry

On Tue, Sep 10, at 5:38 pm, David Kang <dkang at isi.edu> wrote:

> 
> 
> ----- Original Message -----
>> From: "Russell Bryant" <rbryant at redhat.com>
>> To: "David Kang" <dkang at isi.edu>
>> Cc: "OpenStack Development Mailing List" <openstack-dev at lists.openstack.org>
>> Sent: Tuesday, September 10, 2013 5:17:15 PM
>> Subject: Re: [openstack-dev] [nova] [pci device passthrough] fails with "NameError: global name '_' is not defined"
>> On 09/10/2013 05:03 PM, David Kang wrote:
>> >
>> > ----- Original Message -----
>> >> From: "Russell Bryant" <rbryant at redhat.com>
>> >> To: "OpenStack Development Mailing List"
>> >> <openstack-dev at lists.openstack.org>
>> >> Cc: "David Kang" <dkang at isi.edu>
>> >> Sent: Tuesday, September 10, 2013 4:42:41 PM
>> >> Subject: Re: [openstack-dev] [nova] [pci device passthrough] fails
>> >> with "NameError: global name '_' is not defined"
>> >> On 09/10/2013 03:56 PM, David Kang wrote:
>> >>>
>> >>>  Hi,
>> >>>
>> >>>   I'm trying to test pci device passthrough feature.
>> >>> Havana3 is installed using Packstack on CentOS 6.4.
>> >>> Nova-compute dies right after start with error "NameError: global
>> >>> name '_' is not defined".
>> >>> I'm not sure if it is due to misconfiguration of nova.conf or bug.
>> >>> Any help will be appreciated.
>> >>>
>> >>> Here is the info:
>> >>>
>> >>> /etc/nova/nova.conf:
>> >>> pci_alias={"name":"test", "product_id":"7190", "vendor_id":"8086",
>> >>> "device_type":"ACCEL"}
>> >>>
>> >>> pci_passthrough_whitelist=[{"vendor_id":"8086","product_id":"7190"}]
>> >>>
>> >>>  With that configuration, nova-compute fails with the following
>> >>>  log:
>> >>>
>> >>>   File
>> >>>   "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> >>>   line 461, in _process_data
>> >>>     **args)
>> >>>
>> >>>   File
>> >>>   "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py",
>> >>>   line 172, in dispatch
>> >>>     result = getattr(proxyobj, method)(ctxt, **kwargs)
>> >>>
>> >>>   File
>> >>>   "/usr/lib/python2.6/site-packages/nova/conductor/manager.py",
>> >>>   line 567, in object_action
>> >>>     result = getattr(objinst, objmethod)(context, *args, **kwargs)
>> >>>
>> >>>   File "/usr/lib/python2.6/site-packages/nova/objects/base.py",
>> >>>   line
>> >>>   141, in wrapper
>> >>>     return fn(self, ctxt, *args, **kwargs)
>> >>>
>> >>>   File
>> >>>   "/usr/lib/python2.6/site-packages/nova/objects/pci_device.py",
>> >>>   line 242, in save
>> >>>     self._from_db_object(context, self, db_pci)
>> >>>
>> >>> NameError: global name '_' is not defined
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup Traceback (most recent call
>> >>> last):
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/threadgroup.py",
>> >>> line 117, in wait
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup x.wait()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/threadgroup.py",
>> >>> line 49, in wait
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return self.thread.wait()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line
>> >>> 166, in wait
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return self._exit_event.wait()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in
>> >>> wait
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return hubs.get_hub().switch()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177,
>> >>> in switch
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return self.greenlet.switch()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line
>> >>> 192, in main
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup result = function(*args,
>> >>> **kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/service.py",
>> >>> line 65, in run_service
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup service.start()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/service.py", line 164, in
>> >>> start
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup self.manager.pre_start_hook()
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line
>> >>> 805, in pre_start_hook
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> self.update_available_resource(nova.context.get_admin_context())
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line
>> >>> 4773, in update_available_resource
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> rt.update_available_resource(context)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py",
>> >>> line 246, in inner
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return f(*args, **kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py",
>> >>> line 318, in update_available_resource
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup self._sync_compute_node(context,
>> >>> resources)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py",
>> >>> line 347, in _sync_compute_node
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup self._update(context, resources,
>> >>> prune_stats=True)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py",
>> >>> line 420, in _update
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup self.pci_tracker.save(context)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/pci/pci_manager.py", line
>> >>> 126, in save
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup dev.save(context)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 134,
>> >>> in wrapper
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup ctxt, self, fn.__name__, args,
>> >>> kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/conductor/rpcapi.py", line
>> >>> 497, in object_action
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup objmethod=objmethod, args=args,
>> >>> kwargs=kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 85, in
>> >>> call
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return
>> >>> self._invoke(self.proxy.call, ctxt, method, **kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 63, in
>> >>> _invoke
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return cast_or_call(ctxt, msg,
>> >>> **self.kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/proxy.py",
>> >>> line 126, in call
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup result = rpc.call(context,
>> >>> real_topic, msg, timeout)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py",
>> >>> line 139, in call
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return _get_impl().call(CONF,
>> >>> context, topic, msg, timeout)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py",
>> >>> line 794, in call
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> rpc_amqp.get_connection_pool(conf,
>> >>> Connection))
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> >>> line 574, in call
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup rv = list(rv)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> >>> line 539, in __iter__
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup raise result
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup NameError: global name '_' is
>> >>> not
>> >>> defined
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup Traceback (most recent call
>> >>> last):
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> >>> line 461, in _process_data
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup **args)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py",
>> >>> line 172, in dispatch
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup result = getattr(proxyobj,
>> >>> method)(ctxt, **kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/conductor/manager.py", line
>> >>> 567, in object_action
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup result = getattr(objinst,
>> >>> objmethod)(context, *args, **kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 141,
>> >>> in wrapper
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup return fn(self, ctxt, *args,
>> >>> **kwargs)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup File
>> >>> "/usr/lib/python2.6/site-packages/nova/objects/pci_device.py",
>> >>> line
>> >>> 242, in save
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup self._from_db_object(context,
>> >>> self, db_pci)
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup
>> >>> 2013-09-10 12:52:23.774 14749 TRACE
>> >>> nova.openstack.common.threadgroup NameError: global name '_' is
>> >>> not
>> >>> defined
>> >>
>> >> Can you file a bug for this?
>> >>
>> >> Fix here: https://review.openstack.org/45949
>> >>
>> >> --
>> >> Russell Bryant
>> >
>> >
>> >
>> >  Thanks, Russell.
>> >
>> >  The bug is reported.
>> > https://bugs.launchpad.net/nova/+bug/1223559
>> >
>> >  But, another error happens after the patch is applied. "TypeError:
>> >  expected string or buffer".
>> >
>> > ----- log message -----
>> >
>> >   File
>> >   "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> >   line 461, in _process_data
>> >     **args)
>> >
>> >   File
>> >   "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py",
>> >   line 172, in dispatch
>> >     result = getattr(proxyobj, method)(ctxt, **kwargs)
>> >
>> >   File "/usr/lib/python2.6/site-packages/nova/conductor/manager.py",
>> >   line 567, in object_action
>> >     result = getattr(objinst, objmethod)(context, *args, **kwargs)
>> >
>> >   File "/usr/lib/python2.6/site-packages/nova/objects/base.py", line
>> >   141, in wrapper
>> >     return fn(self, ctxt, *args, **kwargs)
>> >
>> >   File
>> >   "/usr/lib/python2.6/site-packages/nova/objects/pci_device.py",
>> >   line 243, in save
>> >     self._from_db_object(context, self, db_pci)
>> >
>> >   File
>> >   "/usr/lib/python2.6/site-packages/nova/objects/pci_device.py",
>> >   line 150, in _from_db_object
>> >     pci_device.extra_info = jsonutils.loads(extra_info)
>> >
>> >   File
>> >   "/usr/lib/python2.6/site-packages/nova/openstack/common/jsonutils.py",
>> >   line 158, in loads
>> >     return json.loads(s)
>> >
>> >   File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
>> >     return _default_decoder.decode(s)
>> >
>> >   File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
>> >     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>> >
>> > TypeError: expected string or buffer
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup Traceback (most recent call last):
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/threadgroup.py",
>> > line 117, in wait
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup x.wait()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/threadgroup.py",
>> > line 49, in wait
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return self.thread.wait()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line
>> > 166, in wait
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return self._exit_event.wait()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in
>> > wait
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return hubs.get_hub().switch()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177,
>> > in switch
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return self.greenlet.switch()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line
>> > 192, in main
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup result = function(*args, **kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/service.py",
>> > line 65, in run_service
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup service.start()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/service.py", line 164, in
>> > start
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup self.manager.pre_start_hook()
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line
>> > 805, in pre_start_hook
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > self.update_available_resource(nova.context.get_admin_context())
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line
>> > 4773, in update_available_resource
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > rt.update_available_resource(context)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py",
>> > line 246, in inner
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return f(*args, **kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py",
>> > line 318, in update_available_resource
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup self._sync_compute_node(context,
>> > resources)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py",
>> > line 347, in _sync_compute_node
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup self._update(context, resources,
>> > prune_stats=True)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/compute/resource_tracker.py",
>> > line 420, in _update
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup self.pci_tracker.save(context)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/pci/pci_manager.py", line
>> > 126, in save
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup dev.save(context)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 134,
>> > in wrapper
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup ctxt, self, fn.__name__, args,
>> > kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/conductor/rpcapi.py", line
>> > 497, in object_action
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup objmethod=objmethod, args=args,
>> > kwargs=kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 85, in
>> > call
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return
>> > self._invoke(self.proxy.call, ctxt, method, **kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 63, in
>> > _invoke
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return cast_or_call(ctxt, msg,
>> > **self.kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/proxy.py",
>> > line 126, in call
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup result = rpc.call(context,
>> > real_topic, msg, timeout)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py",
>> > line 139, in call
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return _get_impl().call(CONF,
>> > context, topic, msg, timeout)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py",
>> > line 794, in call
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup rpc_amqp.get_connection_pool(conf,
>> > Connection))
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> > line 574, in call
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup rv = list(rv)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> > line 539, in __iter__
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup raise result
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup TypeError: expected string or
>> > buffer
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup Traceback (most recent call last):
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py",
>> > line 461, in _process_data
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup **args)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py",
>> > line 172, in dispatch
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup result = getattr(proxyobj,
>> > method)(ctxt, **kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/conductor/manager.py", line
>> > 567, in object_action
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup result = getattr(objinst,
>> > objmethod)(context, *args, **kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 141,
>> > in wrapper
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return fn(self, ctxt, *args,
>> > **kwargs)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/objects/pci_device.py", line
>> > 243, in save
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup self._from_db_object(context,
>> > self, db_pci)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/objects/pci_device.py", line
>> > 150, in _from_db_object
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup pci_device.extra_info =
>> > jsonutils.loads(extra_info)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib/python2.6/site-packages/nova/openstack/common/jsonutils.py",
>> > line 158, in loads
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return json.loads(s)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib64/python2.6/json/__init__.py", line 307, in loads
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup return _default_decoder.decode(s)
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup File
>> > "/usr/lib64/python2.6/json/decoder.py", line 319, in decode
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup obj, end = self.raw_decode(s,
>> > idx=_w(s, 0).end())
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup
>> > 2013-09-10 13:56:35.366 16736 TRACE
>> > nova.openstack.common.threadgroup TypeError: expected string or
>> > buffer
>> 
>> Try this:
>> 
>> diff --git a/nova/objects/pci_device.py b/nova/objects/pci_device.py
>> index a83b8f3..d0a628a 100644
>> --- a/nova/objects/pci_device.py
>> +++ b/nova/objects/pci_device.py
>> @@ -145,7 +145,7 @@ class PciDevice(base.NovaPersistentObject,
>> base.NovaObject):
>> if key != 'extra_info':
>> pci_device[key] = db_dev[key]
>> else:
>> - extra_info = db_dev.get("extra_info")
>> + extra_info = db_dev.get("extra_info", '{}')
>> pci_device.extra_info = jsonutils.loads(extra_info)
>> pci_device._context = context
>> pci_device.obj_reset_changes()
>> 
>> 
>> --
>> Russell Bryant
> 
> 
>  The same error happens.
> The error message says "TypeError: expected string or buffer".

I get the same. Without Russel's patch I get:
"TypeError: 'NoneType' object is not iterable"

I tried a couple of ways to fix the handling of empty extra_info but
couldn't make it work.

I never saw the "NameError: global name '_' is not defined" issue. I wonder
if Bug #1223559 is properly describing the issue? I think it should be
titled "pci passthrough fails trying to handle empty extra_info" or
something like that.

Is anyone looking at this? I feel that the bug should be critical since PCI
passthrough simply does not work at this point.