Hi Gary, <div><br></div><div>Thanks for reporting this.  <div><br></div><div>After some digging, the bug seems to have been introduced by an error in a blanket changeset to switch Nova from using json to jasonutils.  During that translation, a "return" statement was dropped, meaning the serializer always returned Null regardless of what data was pushed in.  See: <a href="https://github.com/openstack/nova/commit/ba76b954e69de56f76f9db5cade1780bc351be67#L5L42">https://github.com/openstack/nova/commit/ba76b954e69de56f76f9db5cade1780bc351be67#L5L42</a></div>

<div><br></div><div>Another bug where having Quantum be part of the gating process would have helped... so we really need to find resources to help get that work complete.  </div><div><br></div><div>Patch here is trivial, so I'll push it and hopefully it can get reviewed quickly by the Nova core team.</div>

<div><br></div><div>Dan</div><div><br></div><div><br></div><div><br><br><div class="gmail_quote">On Thu, May 17, 2012 at 12:24 AM, Gary Kotton <span dir="ltr"><<a href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a>></span> wrote:<br>

<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>

  
    
  
  <div bgcolor="#ffffff" text="#000000">
    Hi,<br>
    Running devstack this morning the following happens:<br>
    <br>
    + /opt/stack/nova/bin/nova-manage network create private <a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a>
    1 256<br>
    2012-05-17 03:22:05 WARNING nova.network.quantum.manager
    [req-5458d926-54d1-4877-9c46-d04035742eaf None None] QuantumManager
    does not use 'bridge' parameter.<br>
    2012-05-17 03:22:05 WARNING nova.network.quantum.manager
    [req-5458d926-54d1-4877-9c46-d04035742eaf None None] QuantumManager
    does not use 'bridge_interface' parameter.<br>
    2012-05-17 03:22:05 DEBUG nova.network.quantum.quantum_connection
    [req-5458d926-54d1-4877-9c46-d04035742eaf None None] Quantum Client
    Request: POST /v1.1/tenants/default/networks.json from (pid=3826)
    do_request /opt/stack/nova/nova/network/quantum/client.py:178<br>
    2012-05-17 03:22:05 DEBUG nova.network.quantum.quantum_connection
    [req-5458d926-54d1-4877-9c46-d04035742eaf None None] Quantum Client
    Reply (code = 400) :<br>
     {"Error": {"message": "The server could not comply with the request
    since it is either malformed or otherwise incorrect.", "code": 400}}
    from (pid=3826) do_request
    /opt/stack/nova/nova/network/quantum/client.py:189<br>
    Command failed, please check log for more info<br>
    2012-05-17 03:22:05 CRITICAL nova
    [req-5458d926-54d1-4877-9c46-d04035742eaf None None] Server 400
    error: {"Error": {"message": "The server could not comply with the
    request since it is either malformed or otherwise incorrect.",
    "code": 400}}<br>
    2012-05-17 03:22:05 TRACE nova Traceback (most recent call last):<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/bin/nova-manage", line 1755, in <module><br>
    2012-05-17 03:22:05 TRACE nova     main()<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/bin/nova-manage", line 1742, in main<br>
    2012-05-17 03:22:05 TRACE nova     fn(*fn_args, **fn_kwargs)<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/bin/nova-manage", line 817, in create<br>
    2012-05-17 03:22:05 TRACE nova     fixed_cidr=fixed_cidr)<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/nova/network/quantum/manager.py", line 232, in
    create_networks<br>
    2012-05-17 03:22:05 TRACE nova     nova_id=nova_id)<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/nova/network/quantum/quantum_connection.py", line
    67, in create_network<br>
    2012-05-17 03:22:05 TRACE nova     resdict =
    self.client.create_network(data, tenant=tenant_id)<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/nova/network/quantum/client.py", line 82, in
    with_params<br>
    2012-05-17 03:22:05 TRACE nova     ret = self.func(instance, *args,
    **kwargs)<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/nova/network/quantum/client.py", line 250, in
    create_network<br>
    2012-05-17 03:22:05 TRACE nova     return self.do_request("POST",
    self.networks_path, body=body)<br>
    2012-05-17 03:22:05 TRACE nova   File
    "/opt/stack/nova/nova/network/quantum/client.py", line 204, in
    do_request<br>
    2012-05-17 03:22:05 TRACE nova     % locals())<br>
    2012-05-17 03:22:05 TRACE nova QuantumServerException: Server 400
    error: {"Error": {"message": "The server could not comply with the
    request since it is either malformed or otherwise incorrect.",
    "code": 400}}<br>
    2012-05-17 03:22:05 TRACE nova <br>
    ++ failed<br>
    ++ local r=1<br>
    ++ set +o xtrace<br>
    <br>
    Thanks<span class="HOEnZb"><font color="#888888"><br>
    Gary</font></span><div><div class="h5"><br>
    <br>
    On 05/17/2012 12:07 AM, Dan Wendlandt wrote:
    <blockquote type="cite"><br>
      <br>
      <div class="gmail_quote">On Wed, May 16, 2012 at 2:01 PM, Sumit
        Naiksatam (snaiksat) <span dir="ltr"><<a href="mailto:snaiksat@cisco.com" target="_blank">snaiksat@cisco.com</a>></span> wrote:<br>
        <blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div link="blue" vlink="purple" lang="EN-US">
            <div>
              <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">The interesting data point here is
                  that this only happens to the default network created
                  when the installation is done via DevStack. For all
                  network created subsequently I am not seeing this
                  issue. I earlier thought that this might have
                  something to do with the exercises script getting
                  executed and possibly leaving a residue. However, in
                  spite of not running the exercises script I am still
                  seeing this issue. It seems that for some reason, the
                  initialize_gateway() was getting called only when a
                  new VM was being created. But here it seems to be
                  getting called before that (thus leading to the
                  creation of the bridge and the gateway devices), not
                  sure why. I am just trying to figure out how I can
                  test the fix with respect to the devstack setup (since
                  this happens only on installation)…</span></p>
            </div>
          </div>
        </blockquote>
        <div><br>
        </div>
        <div>Your explanation makes sense.  I think there was a recent
          comment to automatically initialize all networks when
          nova-network is started, in case nova-network was started on a
          new host (e.g., in an HA configuration) or after a host
          reboot.  Since devstack creates the network using nova-manage
          before starting nova-network, this would trigger the behavior
          your describe.  For more info, see: <a href="https://github.com/openstack/nova/commit/30c8293ea7d4c9464e0139fb488e202fd0947f52" target="_blank">https://github.com/openstack/nova/commit/30c8293ea7d4c9464e0139fb488e202fd0947f52</a></div>


        <div><br>
        </div>
        <div>I suspect you should hit this every time you re-run
          stack.sh, no?  </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div> </div>
        <blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div link="blue" vlink="purple" lang="EN-US">
            <div>
              <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)"></span></p>
              <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)"> </span></p>
              <div style="border-width:medium medium medium 1.5pt;border-style:none none none solid;border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color blue;padding:0in 0in 0in 4pt">
                <div>
                  <div style="border-right:medium none;border-width:1pt medium medium;border-style:solid none none;border-color:rgb(181,196,223) -moz-use-text-color -moz-use-text-color;padding:3pt 0in 0in">
                    <p class="MsoNormal"><b><span style="font-size:10pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10pt;font-family:"Tahoma","sans-serif""> Dan
                        Wendlandt [mailto:<a href="mailto:dan@nicira.com" target="_blank">dan@nicira.com</a>]
                        <br>
                        <b>Sent:</b> Wednesday, May 16, 2012 7:55 AM<br>
                        <b>To:</b> <a href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a><br>
                        <b>Cc:</b> Sumit Naiksatam (snaiksat); <a href="mailto:netstack@lists.launchpad.net" target="_blank">netstack@lists.launchpad.net</a><br>
                        <b>Subject:</b> Re: [Netstack] Fwd: [Openstack]
                        [devstack/quantum] Configuration issue</span></p>
                  </div>
                </div>
                <div>
                  <div>
                    <p class="MsoNormal"> </p>
                    <p class="MsoNormal" style="margin-bottom:12pt">
                       </p>
                    <div>
                      <p class="MsoNormal">On Wed, May 16, 2012 at 1:43
                        AM, Gary Kotton <<a href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a>>
                        wrote:</p>
                      <div>
                        <p class="MsoNormal">
                          Thanks!<br>
                          As far as I recall, and I may certain be wrong
                          here, but in the past I did not see the GW
                          created until the first VM was deployed. Now I
                          am seeing the GW created when the various
                          services are started. </p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal">I have noticed this as
                          well.  I'm not aware of any changes to
                          QuantumManager that had this affect, though
                          its possible someone pushed a change that I
                          didn't notice.  I primarily use the OVS
                          interface-driver and didn't notice any issues,
                          so I suspect Sumit is correct that we can at
                          least work around this with a change to the LB
                          interface-driver (even though the change that
                          caused the issue was likely somewhere else).  </p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal">This is a good example of
                          why we need Quantum integrated into devstack
                          commit-gating, as it will catch this type of
                          complex integration issues.  Anyone have
                          cycles to help push on this?  I'm unlikely to
                          make much progress on it in the next week or
                          two due to the F-1 release.  </p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal">Dan</p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal">
                           </p>
                      </div>
                      <blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">


                        <div>
                          <p class="MsoNormal">Thanks<span style="color:rgb(136,136,136)"><br>
                              <span>Gary</span></span></p>
                          <div>
                            <div>
                              <p class="MsoNormal"><br>
                                <br>
                                On 05/16/2012 10:54 AM, Sumit Naiksatam
                                (snaiksat) wrote: </p>
                              <div>
                                <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">This is
                                    probably not a devstack issue. The
                                    problem is with the LB gateway
                                    driver wherein the IP address is
                                    trying to be set on a bridge device
                                    which already has an IP address. The
                                    check for an existing bridge is
                                    being performed, but the IP address
                                    is being set outside that check.
                                    Ideally, this code should not have
                                    been invoked if the gateway was
                                    already set. Something seems to have
                                    changed in the QuantumManager as
                                    result of which this code is being
                                    invoked again. At any rate, I will
                                    fix the LB gateway driver, and we
                                    will not see this.</span></p>
                                <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)"> </span></p>
                                <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">Thanks,</span></p>
                                <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)">~Sumit.</span></p>
                                <p class="MsoNormal"><span style="font-size:11pt;font-family:"Calibri","sans-serif";color:rgb(31,73,125)"> </span></p>
                                <div style="border-width:medium medium medium 1.5pt;border-style:none none none solid;padding:0in 0in 0in 4pt;border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color blue">


                                  <div>
                                    <div style="border-right:medium none;border-width:1pt medium medium;border-style:solid none none;padding:3pt 0in 0in;border-color:-moz-use-text-color">
                                      <p class="MsoNormal"><b><span style="font-size:10pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10pt;font-family:"Tahoma","sans-serif"">
                                          <a href="mailto:netstack-bounces+snaiksat=cisco.com@lists.launchpad.net" target="_blank">netstack-bounces+snaiksat=cisco.com@lists.launchpad.net</a>
                                          [<a href="mailto:netstack-bounces+snaiksat=cisco.com@lists.launchpad.net" target="_blank">mailto:netstack-bounces+snaiksat=cisco.com@lists.launchpad.net</a>]
                                          <b>On Behalf Of </b>Gary
                                          Kotton<br>
                                          <b>Sent:</b> Monday, May 14,
                                          2012 11:19 PM<br>
                                          <b>To:</b> <a href="mailto:netstack@lists.launchpad.net" target="_blank"><netstack@lists.launchpad.net></a><br>
                                          <b>Subject:</b> [Netstack]
                                          Fwd: [Openstack]
                                          [devstack/quantum]
                                          Configuration issue</span></p>
                                    </div>
                                  </div>
                                  <p class="MsoNormal"> </p>
                                  <p class="MsoNormal"><br>
                                    <br>
                                    -------- Original Message -------- </p>
                                  <table border="0" cellpadding="0" cellspacing="0">
                                    <tbody>
                                      <tr>
                                        <td style="padding:0in" valign="top" nowrap>
                                          <p class="MsoNormal" style="text-align:right" align="right"><b>Subject: </b></p>
                                        </td>
                                        <td style="padding:0in">
                                          <p class="MsoNormal">[Openstack]
                                            [devstack/quantum]
                                            Configuration issue</p>
                                        </td>
                                      </tr>
                                      <tr>
                                        <td style="padding:0in" valign="top" nowrap>
                                          <p class="MsoNormal" style="text-align:right" align="right"><b>Date: </b></p>
                                        </td>
                                        <td style="padding:0in">
                                          <p class="MsoNormal">
                                            Tue, 15 May 2012 09:00:52
                                            +0300</p>
                                        </td>
                                      </tr>
                                      <tr>
                                        <td style="padding:0in" valign="top" nowrap>
                                          <p class="MsoNormal" style="text-align:right" align="right"><b>From: </b></p>
                                        </td>
                                        <td style="padding:0in">
                                          <p class="MsoNormal">Gary
                                            Kotton <a href="mailto:gkotton@redhat.com" target="_blank"><gkotton@redhat.com></a></p>
                                        </td>
                                      </tr>
                                      <tr>
                                        <td style="padding:0in" valign="top" nowrap>
                                          <p class="MsoNormal" style="text-align:right" align="right">
                                            <b>Reply-To: </b></p>
                                        </td>
                                        <td style="padding:0in">
                                          <p class="MsoNormal"><a href="mailto:gkotton@redhat.com" target="_blank">gkotton@redhat.com</a></p>
                                        </td>
                                      </tr>
                                      <tr>
                                        <td style="padding:0in" valign="top" nowrap>
                                          <p class="MsoNormal" style="text-align:right" align="right"><b>Organization:
                                            </b></p>
                                        </td>
                                        <td style="padding:0in">
                                          <p class="MsoNormal">Red Hat</p>
                                        </td>
                                      </tr>
                                      <tr>
                                        <td style="padding:0in" valign="top" nowrap>
                                          <p class="MsoNormal" style="text-align:right" align="right"><b>To: </b></p>
                                        </td>
                                        <td style="padding:0in">
                                          <p class="MsoNormal"><a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a></p>
                                        </td>
                                      </tr>
                                    </tbody>
                                  </table>
                                  <p class="MsoNormal" style="margin-bottom:12pt"> </p>
                                  <pre>Hi,</pre>
                                  <pre>This morning I encountered a problem (which did not happen a few days </pre>
                                  <pre>ago :)). When devstack is launched, with quantum configured, the gateway </pre>
                                  <pre>and bridge devices are created. This causes problems with quantum.</pre>
                                  <pre> </pre>
                                  <pre>For example when devstack is up and running prior to deploying an </pre>
                                  <pre>instance we have:</pre>
                                  <pre> </pre>
                                  <pre>brq744ec2f4-c0 Link encap:Ethernet  HWaddr fa:16:3e:03:a6:55</pre>
                                  <pre>          inet addr:10.0.0.1  Bcast:0.0.0.0  Mask:255.255.255.0</pre>
                                  <pre>          UP BROADCAST MULTICAST  MTU:1500  Metric:1</pre>
                                  <pre>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0</pre>
                                  <pre>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0</pre>
                                  <pre>          collisions:0 txqueuelen:0</pre>
                                  <pre>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)</pre>
                                  <pre> </pre>
                                  <pre>gw-744ec2f4-c0 Link encap:Ethernet  HWaddr fa:16:3e:03:a6:55</pre>
                                  <pre>          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0</pre>
                                  <pre>          UP BROADCAST MULTICAST  MTU:1500  Metric:1</pre>
                                  <pre>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0</pre>
                                  <pre>          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0</pre>
                                  <pre>          collisions:0 txqueuelen:500</pre>
                                  <pre>          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)</pre>
                                  <pre> </pre>
                                  <pre>When an instance is deployed the following happens:</pre>
                                  <pre> </pre>
                                  <pre>2012-05-15 01:59:18 DEBUG nova.utils </pre>
                                  <pre>[req-4d50ed10-46e1-406c-9074-dc45da860365 </pre>
                                  <pre>df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] </pre>
                                  <pre>Running cmd (subprocess): sudo /usr/local/bin/nova-rootwrap ip address </pre>
                                  <pre>add <a href="http://10.0.0.1/24" target="_blank">10.0.0.1/24</a> dev brq744ec2f4-c0 from (pid=4234) execute </pre>
                                  <pre>/opt/stack/nova/nova/utils.py:178</pre>
                                  <pre>2012-05-15 01:59:18 DEBUG nova.utils </pre>
                                  <pre>[req-4d50ed10-46e1-406c-9074-dc45da860365 </pre>
                                  <pre>df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] </pre>
                                  <pre>Result was 254 from (pid=4234) execute /opt/stack/nova/nova/utils.py:194</pre>
                                  <pre>2012-05-15 01:59:18 ERROR nova.rpc.amqp </pre>
                                  <pre>[req-4d50ed10-46e1-406c-9074-dc45da860365 </pre>
                                  <pre>df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] </pre>
                                  <pre>Exception during message handling</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp Traceback (most recent call last):</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/rpc/amqp.py", line 263, in _process_data</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     rval = </pre>
                                  <pre>node_func(context=ctxt, **node_args)</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/network/quantum/manager.py", line 390, in </pre>
                                  <pre>allocate_for_instance</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     network, vif_rec, </pre>
                                  <pre>network['net_tenant_id'])</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/utils.py", line 880, in inner</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     retval = f(*args, **kwargs)</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/network/quantum/manager.py", line 501, in enable_dhcp</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     </pre>
                                  <pre>self.l3driver.initialize_gateway(network_ref)</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/network/l3.py", line 98, in initialize_gateway</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     </pre>
                                  <pre>gateway=(network_ref['gateway'] is not None))</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/network/linux_net.py", line 900, in plug</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     return </pre>
                                  <pre>_get_interface_driver().plug(network, mac_address, gateway)</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/network/linux_net.py", line 1160, in plug</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     run_as_root=True)</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp   File </pre>
                                  <pre>"/opt/stack/nova/nova/utils.py", line 201, in execute</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp     cmd=' '.join(cmd))</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp ProcessExecutionError: </pre>
                                  <pre>Unexpected error while running command.</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp Command: sudo </pre>
                                  <pre>/usr/local/bin/nova-rootwrap ip address add <a href="http://10.0.0.1/24" target="_blank">10.0.0.1/24</a> dev brq744ec2f4-c0</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp Exit code: 254</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp Stdout: ''</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp Stderr: 'RTNETLINK answers: File </pre>
                                  <pre>exists\n'</pre>
                                  <pre>2012-05-15 01:59:18 TRACE nova.rpc.amqp</pre>
                                  <pre>2012-05-15 01:59:18 ERROR nova.rpc.common </pre>
                                  <pre>[req-4d50ed10-46e1-406c-9074-dc45da860365 </pre>
                                  <pre>df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] </pre>
                                  <pre>Returning exception Unexpected error while running command.</pre>
                                  <pre>Command: sudo /usr/local/bin/nova-rootwrap ip address add <a href="http://10.0.0.1/24" target="_blank">10.0.0.1/24</a> </pre>
                                  <pre>dev brq744ec2f4-c0</pre>
                                  <pre>Exit code: 254</pre>
                                  <pre>Stdout: ''</pre>
                                  <pre>Stderr: 'RTNETLINK answers: File exists\n' to caller</pre>
                                  <pre>2012-05-15 01:59:18 ERROR nova.rpc.common </pre>
                                  <pre>[req-4d50ed10-46e1-406c-9074-dc45da860365 </pre>
                                  <pre>df07eec326434b25800f3ebc17202fb3 2cafe0be4d7740098a89be39ffd1b72e] </pre>
                                  <pre>['Traceback (most recent call last):\n', '  File </pre>
                                  <pre>"/opt/stack/nova/nova/rpc/amqp.py", line 263, in _process_data\n    rval </pre>
                                  <pre>= node_func(context=ctxt, **node_args)\n', '  File </pre>
                                  <pre>"/opt/stack/nova/nova/network/quantum/manager.py", line 390, in </pre>
                                  <pre>allocate_for_instance\n    network, vif_rec, </pre>
                                  <pre>network[\'net_tenant_id\'])\n', '  File "/opt/stack/nova/nova/utils.py", </pre>
                                  <pre>line 880, in inner\n    retval = f(*args, **kwargs)\n', '  File </pre>
                                  <pre>"/opt/stack/nova/nova/network/quantum/manager.py", line 501, in </pre>
                                  <pre>enable_dhcp\n    self.l3driver.initialize_gateway(network_ref)\n', '  </pre>
                                  <pre>File "/opt/stack/nova/nova/network/l3.py", line 98, in </pre>
                                  <pre>initialize_gateway\n    gateway=(network_ref[\'gateway\'] is not </pre>
                                  <pre>None))\n', '  File "/opt/stack/nova/nova/network/linux_net.py", line </pre>
                                  <pre>900, in plug\n    return _get_interface_driver().plug(network, </pre>
                                  <pre>mac_address, gateway)\n', '  File </pre>
                                  <pre>"/opt/stack/nova/nova/network/linux_net.py", line 1160, in plug\n    </pre>
                                  <pre>run_as_root=True)\n', '  File "/opt/stack/nova/nova/utils.py", line 201, </pre>
                                  <pre>in execute\n    cmd=\' \'.join(cmd))\n', "ProcessExecutionError: </pre>
                                  <pre>Unexpected error while running command.\nCommand: sudo </pre>
                                  <pre>/usr/local/bin/nova-rootwrap ip address add <a href="http://10.0.0.1/24" target="_blank">10.0.0.1/24</a> dev </pre>
                                  <pre>brq744ec2f4-c0\nExit code: 254\nStdout: ''\nStderr: 'RTNETLINK answers: </pre>
                                  <pre>File exists\\n'\n"]</pre>
                                  <pre> </pre>
                                  <pre> </pre>
                                  <pre>My guess is that this can be solved with the nova configuration. Sadly I </pre>
                                  <pre>am not too familiar with these. Any help would be great.</pre>
                                  <pre>Thanks</pre>
                                  <pre>Gary</pre>
                                  <pre> </pre>
                                  <pre>_______________________________________________</pre>
                                  <pre>Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a></pre>
                                  <pre>Post to     : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a></pre>
                                  <pre>Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a></pre>
                                  <pre>More help   : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a></pre>
                                </div>
                              </div>
                              <p class="MsoNormal"> </p>
                            </div>
                          </div>
                        </div>
                        <p class="MsoNormal" style="margin-bottom:12pt"><br>
                          --<br>
                          Mailing list: <a href="https://launchpad.net/%7Enetstack" target="_blank">https://launchpad.net/~netstack</a><br>
                          Post to     : <a href="mailto:netstack@lists.launchpad.net" target="_blank">netstack@lists.launchpad.net</a><br>
                          Unsubscribe : <a href="https://launchpad.net/%7Enetstack" target="_blank">https://launchpad.net/~netstack</a><br>
                          More help   : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a></p>
                      </blockquote>
                    </div>
                    <p class="MsoNormal"><br>
                      <br clear="all">
                    </p>
                    <div>
                      <p class="MsoNormal">
                         </p>
                    </div>
                    <p class="MsoNormal">-- <br>
                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
                      Dan Wendlandt </p>
                    <div>
                      <p class="MsoNormal">Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a></p>
                      <div>
                        <p class="MsoNormal">twitter: danwendlandt<br>
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~</p>
                      </div>
                    </div>
                    <p class="MsoNormal"> </p>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </blockquote>
      </div>
      <br>
      <br clear="all">
      <div>
        <br>
      </div>
      -- <br>
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
      Dan Wendlandt 
      <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br>
        <div>twitter: danwendlandt<br>
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
        </div>
      </div>
      <br>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br><br clear="all"><div><br></div>-- <br>~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Dan Wendlandt <div>Nicira, Inc: <a href="http://www.nicira.com" target="_blank">www.nicira.com</a><br><div>twitter: danwendlandt<br>

~~~~~~~~~~~~~~~~~~~~~~~~~~~<br></div></div><br>
</div></div>