[openstack-dev] [nova] massive number of new errors in logs with oslo.messaging

David Kranz dkranz at redhat.com
Mon Feb 3 15:10:04 UTC 2014


On 02/03/2014 07:36 AM, Monty Taylor wrote:
> On 02/02/2014 08:33 PM, Jay Pipes wrote:
>> On Sun, 2014-02-02 at 07:13 -0500, Sean Dague wrote:
>>> Just noticed this at the end of a successful run:
>>>
>>> http://logs.openstack.org/15/63215/13/check/check-tempest-dsvm-full/2636cae/console.html#_2014-02-02_12_02_44_422 
>>>
>>>
>>> It looks like the merge of oslo.messaging brings a huge amount of false
>>> negative error messages in the logs. Would be good if we didn't ship
>>> icehouse with this state of things.
>>
>> Agreed.
>>
>> And the error messages, which look like this:
>>
>> Returning exception Unexpected task state: expecting [u'scheduling',
>> None] but the actual state is deleting to caller
>>
>> don't make sense -- at least in the English language.
>>
>> What does "the actual state is deleting to caller" mean?
>
> I'm going to try to translate, with no real context:
>
> I'm trying to return a state to the caller. While trying to do that, 
> I've encountered an unexpected task state. I was expecting the state 
> to be [u'scheduling', None'] but instead I seem to have discovered 
> that the state was in fact, deleting. I'm freaked out by this, so 
> instead of returning a task state to the caller, I'm going to return 
> an exception.
>
> If the above is correct, I'd say that this is pretty terrible. We 
> probably don't need to log that we're returning a state. That's crazy 
> detailed. If we _do_ encounter an exception, we probably shouldn't 
> encapsulate the exception state and try to return it as the state text.
>
> That said - I actually have no idea what's going on here. :)
I don't either, but this is why we started failing gate jobs that put 
unexpected errors in the logs even though all tests "passed". 
Unfortunately I hear that the log-errors-failing-jobs feature was 
reverted as part of the recent process of restoring the gate to health. 
Is restoring that behavior on the radar screen?

  -David
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list