[Openstack] Compute Node Down!

Tom Fifield fifieldt at unimelb.edu.au
Thu Sep 20 05:05:35 UTC 2012


On 20/09/12 13:50, Vishvananda Ishaya wrote:
> **
>
> On Wed, Sep 19, 2012 at 4:03 AM, Wolfgang Hennerbichler
> <wolfgang.hennerbichler at risc-software.at
> <mailto:wolfgang.hennerbichler at risc-software.at>> wrote:
>
>     Hello Folks,
>
>     Although it seems a pretty straightforward scenario I have a hard
>     time finding documentation on this.
>     One of my compute nodes broke down. All the instances are on shared
>     storage, so no troubles here, but I don't know how to tell openstack
>     that the VM should be deployed on another compute node. I tried
>     fiddling around in the mysql-db with no success.
>     Any help is really appreciated.
>
>     Wolfgang




== Dead compute host ==
Working with the host information
<pre>
i-000015b9 at3-ui02 running nectarkey (376, np-rcc54) 0 m1.xxlarge 
2012-06-19T00:48:11.000Z 115.146.93.60
</pre>

# review the status of the host using the nova database, some of the 
important information is highlighted below.
<pre>
SELECT * FROM instances WHERE id = CONV('15b9', 16, 10) \G;
*************************** 1. row ***************************
               created_at: 2012-06-19 00:48:11
               updated_at: 2012-07-03 00:35:11
               deleted_at: NULL
...
                       id: 5561
...
              power_state: 5
                 vm_state: shutoff
...
                 hostname: at3-ui02
                     host: np-rcc54
...
                     uuid: 3f57699a-e773-4650-a443-b4b37eed5a06
...
               task_state: NULL
...
</pre>

Update the vm's compute host.
<pre>
UPDATE instances SET host = 'np-rcc46' WHERE uuid = 
'3f57699a-e773-4650-a443-b4b37eed5a06';
</pre>

Update the libvirt xml

* change the DHCPSERVER value to the host ip address.
* possibly the VNC IP if it isn't already 0.0.0.0

Dump a copy of a nwfilter to use as a template for creating the missing 
nwfilter.
<pre>
virsh nwfilter-list
vrish nwfilter-dumpxml nova-instance-instance-.....
</pre>

Example of the template file
<pre>
<filter name='nova-instance-instance-00001cc6-fa163e003b43' chain='root'>
   <uuid>d5f6f610-d0b8-4407-ae00-5dabef80677a</uuid>
   <filterref filter='nova-base'/>
</filter>

</pre>
The filter name value is available from the instances.xml file 
(<filterref filter="nova-instance-instance-00001cc6-fa163e003b43">).
*Note the filter name must be exact!
Generate a new uuid and replace it at the uuid value.

Update filter to match id from instance xml
<pre>
virsh nwfilter-define /tmp/filter.xml
virsh define libvirt.xml
virsh list --all
</pre>

Kill all dnsmasq and restart nova services.
<pre>
killall dnsmasq; service nova-network restart; service nova-compute restart
</pre>

Start the vm
<pre>
virsh start instance-00000
</pre>

On the nova DB
<pre>
UPDATE instances SET vm_state = 'active', power_state = 1 WHERE uuid = 
'3f57699a-e773-4650-a443-b4b37eed5a06';
</pre>




More information about the Openstack mailing list