Open Stack

I noticed something similar when deploying Keystone using nginx in the lab, and pretty sure I fixed it by setting  uwsgi_ignore_client_abort to on.
http://nginx.org/en/docs/http/ngx_http_uwsgi_module.html

In addition to that flag I also have
client_header_buffer_size 64k;

uwsgi_buffer_size         8k;
uwsgi_read_timeout        600;
uwsgi_send_timeout        600;

Best Regards, Erik Olof Gunnar Andersson

-----Original Message-----
From: Thomas Goirand <zigo at debian.org> 
Sent: Wednesday, May 2, 2018 11:28 AM
To: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] Problems with all OpenStack APIs & uwsgi with Content-Lenght and connection reset by peer (ie: 104)

On 05/02/2018 10:25 AM, Chris Dent wrote:
> On Wed, 2 May 2018, Thomas Goirand wrote:
> 
>> What was disturbing was that, doing the same request with curl worked 
>> perfectly. Even more disturbing, it looked like I was having the 
>> issue nearly always in virtualbox, but not always in real hardware, 
>> where it sometimes worked.
> 
> What was making the request in the first place? It fails in X, but 
> works in curl. What is X?

For example, nova-compute querying nova-placement-api.
Another example: openstackclient. It happened to me trying to configure keystone when running puppet-openstack, for example, but on the command line directly as well, simply trying to add users, projects, etc. This looks like to me as a general problem in all of the OpenStack WSGI applications.

>> Anyway, finally, I figured out that adding:
>>
>> --rem-header Content-Lenght
> 
> You added this arg to what?

As a parameter to uwsgi, so that it removes the Content-Lenght that the WSGI application sends.

>> This however, looks like a workaround rather than a fix, and I wonder 
>> if there's a real issue somewhere that needs to be fixed in a better 
>> way, maybe in openstackclient or some other component...
> 
> Yeah, it sounds like something could be setting a bad value for the 
> content length header and uwsgi is timing out while trying to read 
> that much data (meaning, it is believing the content-length header) 
> but there isn't anything actually there.
> 
> Another option is that there are buffer size problems in the uwsgi 
> configuration but it's hard to speculate because it is not clear what 
> requests and tools you're actually talking about here.

When attempting to google for the issue, I saw a lot of people that had this problem fixed by adding --buffer-size 65535, as the default 4k header of uwsgi was not enough. I also have this option set, as it seems a reasonable thing to have, but that was not enough to fix the problem.
Only the --rem-header thing did.

If you want to try, you can simply use the stretch-queens.debian.net repository with Glance (or simply Debian Sid), and edit /etc/glance/glance-api-wsgi.ini to change the uwsgi parameters (I've just switched Glance to uwsgi, since it now works...). I haven't checked with Glance, but since I saw the problem with nova-placement-api, cinder-api and keystone, I don't see why it wouldn't happen there.

Cheers,

Thomas Goirand (zigo)

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev