[openstack-dev] [nova] loss of WSGI request logs with request-id under uwsgi/apache
Sean Dague
sean at dague.net
Thu Jul 20 11:06:21 UTC 2017
On 07/19/2017 09:46 PM, Matt Riedemann wrote:
> On 7/19/2017 6:16 AM, Sean Dague wrote:
>> I was just starting to look through some logs to see if I could line up
>> request ids (part of global request id efforts), when I realized that in
>> the process to uwsgi by default, we've entirely lost the INFO wsgi
>> request logs. :(
>>
>> Instead of the old format (which was coming out of oslo.service) we get
>> the following -
>> http://logs.openstack.org/97/479497/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/5a0fb17/logs/screen-n-api.txt.gz#_Jul_19_03_44_58_233532
>>
>>
>>
>> That definitely takes us a step backwards in understanding the world, as
>> we lose our request id on entry that was extremely useful to match up
>> everything. We hit a similar issue with placement, and added custom
>> paste middleware for that. Maybe we need to consider a similar thing
>> here, that would only emit if running under uwsgi/apache?
>>
>> Thoughts?
>>
>> -Sean
>>
>
> I'm noticing some other weirdness here:
>
> http://logs.openstack.org/65/483565/4/check/gate-tempest-dsvm-py35-ubuntu-xenial/9921636/logs/screen-n-sch.txt.gz#_Jul_19_20_17_18_801773
>
>
> The first part of the log message got cut off:
>
> Jul 19 20:17:18.801773 ubuntu-xenial-infracloud-vanilla-9950433
> nova-scheduler[22773]:
> -01dc-4de3-9da7-8eb3de9e305e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active'),
> 'a4eba582-075a-4200-ae6f-9fc7797c95dd':
No, it's the log message exceeded buffer limits in systemd journal, and
was split across lines. It starts 2 more lines up.
-Sean
--
Sean Dague
http://dague.net
More information about the OpenStack-dev
mailing list