Open Stack

Johnathan,

this would be easier to debug with a nova-compute log. Are you willing
to post one somewhere that people could take a look at?

Thanks,
Michael

On Thu, Aug 8, 2013 at 7:35 AM, Jonathan Proulx <jon at jonproulx.com> wrote:
> Hi All,
>
> Apologies to those who saw this on the operators list earlier, there is a
> bit of new info here & having gotten no response there thought I'd take it
> to a wider audience...
>
>
> I'm almost through my grizzly upgrade.  I'd upgraded everything except
> nova-compute before upgrading that (ubuntu 12.04 cloud archieve pkgs).
>
> On most nodes the nova-compute service upgraded and restarted properly, but
> on some it imediately exits with:
>
> CRITICAL nova [-] 'instance_type_memory_mb'
>
> It would seem like this is https://code.launchpad.net/bugs/1161022 but the
> fix for that was released in March and I've verified is in the packaged
> version I'm using.
>
> The referenced bug involves the DB migration only updating non-deleted
> instances in the instance-system-metatata table and the patch skips the
> lookups that are broken (and irrelevant) for deleted instances.
>
> Tracing the DB calls from the host shows it is trying to do lookups for
> instances that were deleted last October, which is a bit surprising as it's
> run thousands of instances since & it's not looking those up.
>
> It is note worthy that that is around the time I upgraded from Essex ->
> Folsom so it's possible their state  is weirder than most having run through
> that update.
>
> There were directories for the instances in question in
> /var/lib/nova/instances, so I thought "Aha!" and moved them, but on restart
> I still get the same failure and same DB query for the old instances. Where
> is nova getting the idea it should look these up & how can I stop it?
>
> I've go so far as to generate instance_type_<foo> entries in the
> instance_system_metadata table  for all instances ever on my deployment
> (about 500k) but I still only have the cryptic "CRITICAL nova [-]
> 'instance_type_memory_mb'" error and a failure to start, so clearly I'm
> casing the wrong problem some how.
>
> Help?
> -Jon
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>



-- 
Rackspace Australia