Open Stack

On 3 December 2014 at 11:58, Don Waterloo <don.waterloo at gmail.com> wrote:

> I am having a problem that I hope someone can comment on.
>
> Periodically, an instance ends up w/ 0 rows in 'instance_info_caches' in
> the nova database.
>
> as a consequence, when i do 'nova list', it ends up without knowing
> anything about the networks. The instance is allocated an IP, has booted,
> is able to use that IP. Neutron owns the port for it, all is good from that
> standpoint, its just nova knows nothing about it.
>
> Is 'info_caches' something that is truly a cache? it seems the only known
> repository.
>


Sorry to follow up my own email, but... is anyone else hitting this? I'm
getting more than just a 'no ip in nova list' symptom, once in a while some
instance ends up w/ 0 bridges in its virsh xml file. What happens is it
comes up normally, all is good and happy. but then some number of day(?)
later, it ends up with no source bridges, no interfaces, and a [] for a
'network_info' field in the instance_info_caches table.

Any idea how this could happen? Its juno on Ubuntu. In ~10K instances
started/stopped since ~jan 1, I now have 15 in this state, so its not super
common. This symptom is more severe, so I cannot live with it.

A reboot does not solve, nor does rebuild (it rebuilds from this info).
Neutron still says the instance is connected, but nova gets it wrong.

 select * from instance_info_caches where network_info = '[]' and deleted =
0;
+---------------------+---------------------+------------+------+--------------+--------------------------------------+---------+
| created_at          | updated_at          | deleted_at | id   |
network_info | instance_uuid                        | deleted |
+---------------------+---------------------+------------+------+--------------+--------------------------------------+---------+
| 2014-11-03 21:47:44 | 2014-11-03 21:48:05 | NULL       | 4762 | []
    | 6996aa1c-7c05-4e36-a86e-d45f7af14352 |       0 |
 ...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150119/4927dc7e/attachment.html>