<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On 25 Apr 2014, at 12:30 am, Chris Behrens <<a href="mailto:cbehrens@codestud.com">cbehrens@codestud.com</a>> wrote:</div><br><blockquote type="cite"><meta http-equiv="Content-Type" content="text/html charset=windows-1252"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><blockquote type="cite"><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">So, does your nova-conductor not have object version 1.13? As I was trying to get at in a previous reply, I think the only way this can possibly work is that you have Icehouse nova-conductor running in ALL cells.</div></blockquote><div><br></div><div>OK so in my compute cell I am now running an Icehouse conductor. Everything else is Havana including the DB version.</div><div><br></div><div>This actually seems to make all the things that didn’t work now work. However it also means that the thing that did work (booting an instance) no longer works.</div><div>This is an easy fix and just requires nova-conductor to call the run_instance scheduler rpcapi method with version 2.9 as opposed the icehouse version 3.0.</div><div>I don’t think anything has changed here so this might be an easy fix that could be pushed upstream. It just needs to change the scheduler rpcapi to be aware what version it can use.</div><div>I changed the upgrade_levels scheduler=havana but that wasn’t handled by the scheduler rpcapi and just gave a version not new enough exception.</div><div><br></div><div>I think I’m making progress…..</div></div></blockquote><div><br></div><div>Cool. So, what is tested upstream is upgrading everything except nova-compute. You could try upgrading nova-scheduler as well. </div></div></blockquote><div><br></div><div>Yeah that fixes it although I don’t think that’s needed. Ideally what I want to do is upgrade our API cell to Icehouse then one at a time upgrade each of our compute cells.</div><div>This is possible in 2 ways:</div><div><br></div><div>1. Hack the object_backport code in havana to handle icehouse objects (and a few other minor things I mentioned earlier)</div><div>2. Run an Icehouse conductor in the compute cell </div><div><br></div><div>Still figuring out the best way.</div><br><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div><div>Although, I didn’t think we had any build path going through conductor yet. Do you happen to have a traceback from that? (Curious what the call path looks like)</div></div></div></blockquote><div><br></div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Specified RPC version cap, 2.9, is too low. Needs to be higher than 3.0.</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/.venv/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher incoming.message))</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/.venv/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/.venv/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/nova/conductor/manager.py", line 797, in build_instances</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher legacy_bdm_in_spec=legacy_bdm)</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/nova/scheduler/rpcapi.py", line 116, in run_instance</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher cctxt.cast(ctxt, 'run_instance', **msg_kwargs)</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/.venv/local/lib/python2.7/site-packages/oslo/messaging/rpc/client.py", line 130, in cast</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher self._check_version_cap(msg.get('version'))</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher File "/opt/nova-icehouse/.venv/local/lib/python2.7/site-packages/oslo/messaging/rpc/client.py", line 115, in _check_version_cap</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher version_cap=self.version_cap)</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">2014-04-25 08:28:35.975 1171 TRACE oslo.messaging.rpc.dispatcher RPCVersionCapError: Specified RPC version cap, 2.9, is too low. Needs to be higher than 3.0.</div><div style="margin: 0px; font-size: 11px; font-family: Menlo;"><br></div><div style="margin: 0px; font-size: 11px; font-family: Menlo;"><br></div><div style="margin: 0px; font-size: 11px; font-family: Menlo;">Sam</div></div></body></html>