[Openstack] instance_info_caches table, nova not populating for some instances

Michael Still mikal at stillhq.com
Mon Jan 19 20:40:52 UTC 2015


I've never heard of anything like this.

What release of OpenStack are you running? What hypervisor driver?

Thanks,
Michael

On Tue, Jan 20, 2015 at 7:46 AM, Don Waterloo <don.waterloo at gmail.com> wrote:
>
>
> On 3 December 2014 at 11:58, Don Waterloo <don.waterloo at gmail.com> wrote:
>>
>> I am having a problem that I hope someone can comment on.
>>
>> Periodically, an instance ends up w/ 0 rows in 'instance_info_caches' in
>> the nova database.
>>
>> as a consequence, when i do 'nova list', it ends up without knowing
>> anything about the networks. The instance is allocated an IP, has booted, is
>> able to use that IP. Neutron owns the port for it, all is good from that
>> standpoint, its just nova knows nothing about it.
>>
>> Is 'info_caches' something that is truly a cache? it seems the only known
>> repository.
>
>
>
> Sorry to follow up my own email, but... is anyone else hitting this? I'm
> getting more than just a 'no ip in nova list' symptom, once in a while some
> instance ends up w/ 0 bridges in its virsh xml file. What happens is it
> comes up normally, all is good and happy. but then some number of day(?)
> later, it ends up with no source bridges, no interfaces, and a [] for a
> 'network_info' field in the instance_info_caches table.
>
> Any idea how this could happen? Its juno on Ubuntu. In ~10K instances
> started/stopped since ~jan 1, I now have 15 in this state, so its not super
> common. This symptom is more severe, so I cannot live with it.
>
> A reboot does not solve, nor does rebuild (it rebuilds from this info).
> Neutron still says the instance is connected, but nova gets it wrong.
>
>  select * from instance_info_caches where network_info = '[]' and deleted =
> 0;
> +---------------------+---------------------+------------+------+--------------+--------------------------------------+---------+
> | created_at          | updated_at          | deleted_at | id   |
> network_info | instance_uuid                        | deleted |
> +---------------------+---------------------+------------+------+--------------+--------------------------------------+---------+
> | 2014-11-03 21:47:44 | 2014-11-03 21:48:05 | NULL       | 4762 | []
> | 6996aa1c-7c05-4e36-a86e-d45f7af14352 |       0 |
>  ...
>
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>



-- 
Rackspace Australia




More information about the Openstack mailing list